How to Write a Testing Framework

Sometimes, test frameworks seem to be a dime a dozen.  There are many commercial and homegrown frameworks.  All have the same general goal – make automated testing easier.  This is an awesome goal.  When done well, it enables testers to focus on high-value work and lets the framework take care of low-value stuff (important but low value for people to do).

Over the course of the several posts, I will share the story of a services testing framework that several members of my test team and I wrote (with some technical and design help from friends and well wishers).  We started with a few goals and principles and we are driving toward an open-source release of the tool.

To give you a sneak peak, here is the current start page of Cham (there is more than a start page -- we have  a working system with high levels of abstraction and fast, scalable performance). 

Start page of Cham

From this view, you can see some of the things that are important to us when we think about frameworks.

• Separate the technical stuff from the tests.  This is the difference between the templates section and the build tests section.  Hide technical stuff where you can and build awesome tests using business words (also known as people words).  Test built using people words are easier to share, to discuss, and to maintain.
• Focus on making everything understandable by a wide audience.
• Results are easy to share with people who care.  In Cham, we do this a couple ways.  One is by simply making it easy to pass specific results to people (our pages are RESTful and you can simply give someone a URL). Another way (not seen here) is by making the tests work with the developer workflow.  For example, programmers can use Cham to test services before code is checked in.
• Make it easily accessible.  You can’t see this here, but everything is done with open-source code (go Ruby) and libraries so there are no licensing issues limiting access.
• Make it look nice – this gives users more confidence in the tool.  And it makes you feel better about yourself.  We can all use that.