No matter which approach you take to testing, the most important thing is to actually have a set of automated tests and to strive for a level of coverage that gives you benefits that will far outweigh the costs.
It is unrealistic, and probably not worth the cost, to achieve 100% test coverage of your source code. For new code and projects that are new to automated testing, 60 to 80% code coverage for your tests is a reasonable target, although seasoned testers will strive for 80 to 95% coverage. There are tools that will examine your source code and tests, and then tell you what your test coverage is (for example, Emma and Cobertura for Java, and NCover for .NET).
Legacy code is a different story. We have more to say about legacy code later in this chapter. For now, suffice it to say that legacy code will almost certainly start with 0% test coverage. The strategy with legacy code is to get a test harness in place, and then incrementally add tests over time—creating tests for any code that you change or enhance. This low-overhead approach will increase coverage over time and will also increase the quality and malleability of the legacy codebase.
The cardinal rule of automated testing is that each individual test must be independent of any other test. Stated another way, no test can depend on the results of a previous test. If you adhere to these rules, tests can be run in any order, and any individual test can be run by itself.
Every test suite has its own setup and teardown function. The setup function’s job is to initialize the test environment for the suite’s tests. This includes preparing any data (data files, in-memory data, database tables, etc.) so that it is in a known, pristine state. The setup function also initializes any required libraries, subsystems, or services (for unit tests, this can be kept to a minimum by using mocks and stubs).
It is important to know that the test suite’s setup is called multiple times, once before each test in the suite. This ensures a pristine test environment at the start of each test. Correspondingly, the teardown function is called after each test in the suite to clean up and release any resources allocated or initialized by setup.
Unit testing a database-driven application is often a sticking point for novice testers, and the standard canned advice is not very helpful. The traditional answer is to create a mock for your database that lets you run your tests without a database. This only works for very simple schemas and very simple tests.
In the vast majority of cases, the database schema and application are too complicated to create a mock at a reasonable cost. This means that an alternative approach must be used. Keep in mind, however, that no matter what approach is taken, the data environment must be initialized to a known state before each test.
A common architectural approach is to design the application with a data abstraction layer (DAL) that isolates the application from the fact that a database is being used. The application calls the DAL for the logical data it wants, and the DAL queries the database to get the data. This makes it easy to create alternative DALs that go elsewhere for the data (like local files) or return a canned response (in the case of a mock).
If a DAL is not feasible, you might have to use a real database in your testing. This means that you need a test database and a test set of data that you can load into the database before each test. Of course, you’ll want to do this as efficiently as possible so that your tests run quickly. If your database supports nested transactions, you can load the test data into the database once, start a new transaction before each test, and then do a rollback after each test finishes.
There are tools that can help you test with a database (for example, the open source tool DbUnit).
In TDD, you create the tests for each feature of an application before you develop the feature itself. Initially, the tests will fail because the code implementing the feature has not yet been written. You then proceed to implement the feature. You will know when you are done because the failing tests will pass.
Technically, you should always write the test first, before implementing the code. However, many projects are lenient on this rule, and the tests are often written after the feature has been implemented. This is particularly true when the implementation was the result of some exploratory programming.
Purists who want to make absolutely clear that they always write their tests first may say that they practice test-first development (TFD). That said, there are also some benefits to writing tests first that can make it worth striving for.
By writing tests first, you actually view the feature’s API through the eyes of the user of that API. More importantly, you do so before that API has been physically created. This means that you will discover problems with the usability of the API while it is easiest to change. The result is usually a significantly better internal design.
The phrase “Red, Green, Refactor” is heard often in Agile and Lean development circles. It summarizes the endless repetition of the iterative work cycle of TDD until the project is finished:
- Red
This refers to the state where you have written the test for a new feature, but you haven’t yet written the implementing code. When you run the tests, they will fail, and a GUI test runner would show a “red” status.
- Green
You’ve written just enough code to get the tests to pass. At this point, a GUI test runner would show a “green” status.
- Refactor
You designed the code to get the feature working as quickly as possible, but it may not be the best code. In this step you refactor the code to improve its structure.
Refactoring is an essential step for maintaining a high-quality codebase. To refactor means to change the code’s internal structure and implementation without changing its external behavior. You can find entire books on the subject of refactoring. See Appendix A for a list of recommended resources.
The traditional wisdom is that a legacy codebase slowly deteriorates over time. Changes and feature additions become increasingly costly and time-consuming to implement (you touch one thing, and 10 other things break!). Eventually, the codebase becomes so fragile that no one wants to touch it, so you consider only changes that are absolutely essential. The next time the code requires a major change, you tell management that it would be cheaper to throw it away and start over again from scratch.
It doesn’t have to be that way. Instead of slowly rotting away, a legacy codebase can instead be steadily and incrementally improved so that it gets better over time. Better means easier, safer, and less costly to change. The vehicle of this miracle is (drumroll, please...) automated tests!
Michael Feathers wrote the book (literally) on how to get a legacy codebase under test. His book, Working Effectively with Legacy Code (Prentice Hall, 2004), is a must-read for anyone who works with legacy code (at one time or another, this applies to all of us). Feathers has a very simple definition of legacy code: “Legacy code is any code for which automated tests do not exist.”
He goes to great lengths to justify this definition. If we accept it, then many of the developers reading this book probably wrote some legacy code just yesterday!
It is not easy to retrofit tests onto a legacy codebase. First, you can’t just stop all other work and undertake a massive effort to add tests across an entire codebase. (Even if you could convince management to fund it, this kind of boring drudge work would be doomed to failure.) However, you can take an incremental approach where you add tests to any area of the code that you change. Even so, you will likely encounter many other problems trying to test code that was not designed to be testable.
This is where Feathers’s book becomes essential reading. It is basically an encyclopedia of the problems you will run into while retrofitting tests onto legacy code, and it includes a series of techniques for dealing with each type of problem.
BDD is at the current cutting edge of the automated testing field. As such, fewer tools, frameworks, and other resources are available. At the same time, BDD promises some of the greatest benefits of any automated testing technique. Even if you can’t currently practice BDD, it is a field worth keeping up with.
BDD takes the behavior testing that we presented earlier and makes it the driving force behind design and implementation. In BDD, you write the requirements for a feature as one or more behavior tests, before any detailed design or implementation.
Since behavior tests are readable by the business stakeholders, they can be considered to be the official product requirements. The business stakeholders can sign off on them (and approve subsequent changes). Because computers can run them, they become executable specifications. Finally, once the system has been implemented, they also become executable acceptance tests.
Get The Art of Lean Software Development now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
