Never do manually anything that can be automated.
Mistake-proofing is a fundamental Lean concept. It is a central part of producing a quality product and reducing waste (by eliminating rework). This is just as true in software development as it is in manufacturing. Automated testing is the primary means of mistake-proofing in software development.
Not coincidentally, automated testing is also a cornerstone of the Agile methodologies. Automated testing is so ubiquitous in the Agile world that it is just assumed that no one would even consider writing new code without automated tests. Although this is not quite as true with legacy code, even there the trend is toward increasing use of automated testing.
Automated testing is a pretty broad term. We are using it here as an umbrella term to refer to all kinds of testing: unit testing, integration testing, acceptance testing, executable specifications testing, performance testing, load testing, story testing, test-driven development, test-first development, behavior-driven development, etc.
Each of these different types of testing have a particular focus, but they all have the following in common:
The tests are created manually by developers
They can all be run automatically, without human intervention
They are run by some type of test harness
Test failures are detected automatically
The developer is notified when such failures occur
Automated testing supports three of the Lean software development principles described in Chapter 2:
Eliminate waste
Build quality in
Create knowledge
Automated testing helps eliminate the waste of rework that occurs when defects slip through to later phases of the software development lifecycle. Such escaped defects are particularly expensive (in time, money, and reputation) when they make it all the way to deployment.
Automated testing is a primary method for building in quality. A codebase with a full suite of automated tests is self-checking and self-validating. This helps reduce the likelihood that undetected errors will be introduced into the software.
Finally, automated tests serve as living documentation on how to actually use the APIs of the codebase. This creates firsthand knowledge that developers actually trust because it is guaranteed to be accurate every time the test suite successfully executes.
There are numerous benefits to automated testing, and we cover them throughout this chapter, but let’s start with the basics: productivity and quality. The quickest way to simultaneously increase your productivity and the quality of your software is by having a full suite of automated tests. This has been borne out through personal experience and independent studies, including a paper that Hakan Erdogmus presented at the Proceedings of the IEEE Transactions on Software Engineering (2005) titled, “On the Effectiveness of the Test-First Approach to Programming.”
When a project lacks a suite of automated tests, developers are very cautious about making changes or additions, especially if they are not very familiar with the code. This means that a lot of time is spent studying code that is going to be changed, and studying the use of that code throughout the entire codebase. Even then, developers may still feel like they’ve missed something important. This adds a lot of time to both the development of new features and the fixing of problems.
When your project has a suite of automated tests, it acts like a safety net for the developers. Rather than spending an inordinate amount of time studying and understanding the target code, the developer can simply implement the feature (or fix the problem) with a simpler understanding of just the immediate target code. She can safely do so because she knows that any unforeseen adverse consequences will be caught by the automated tests.
When you can confidently add to or modify the code without an excessive amount of research, you can save a substantial amount of time. In addition, since problems are immediately detected and corrected, you can eliminate expensive rework that would otherwise occur if the problems were detected at a much later phase in the project.
Finally, when a developer wants to know how a piece of code works, she will usually avoid any detailed documentation because experience has shown that such documentation is usually out-of-date and, therefore, wrong. This is why developers prefer to study the code itself. Tests effectively document how pieces of code should be used, and they can be proved to be correct and up-to-date simply by running them. Because of this, developers come to rely on tests as reliable, working documentation.
A typical excuse for not creating automated tests is that the deadlines are too tight and “we cannot spend extra time writing tests.” This is a myth and a classic example of shortsighted thinking. Yes, it does take time to write these tests, but this time is more than offset by the savings of eliminating rework, reducing the cost of debugging, and reducing the fear of changing the code. Automated tests actually help you meet your deadlines!
An unexpected benefit of automated testing is that it actually improves the design of the code being tested. There are two primary reasons for this. First, when you write tests against your API, any deficiencies in the design of that API become painfully obvious because you have just become a user of that API. If you are using test-first development, you write the tests even before writing the code. Since you are using your API before it is even implemented, it is very easy to simply change the API or design to address the deficiencies. If the tests are being written after the code, minor deficiencies are usually overlooked, but major deficiencies will still trigger a redesign.
Second, when you are creating automated tests as a part of your development process, you quickly learn to design the code in a manner that ensures that it is testable. Testable code encourages the same good attributes that design and architecture experts have been preaching for years: loose coupling, minimized dependencies, and encapsulation.
A recent trend in automated testing is to write requirements in the form of executable specification tests. New testing frameworks (RSpec, JBehave, NSpec, Spector, etc.) let you write tests that describe the desired system behavior in a manner that is readable by business stakeholders and by developers. The ability to execute these tests eliminates the disconnect between requirements and code. It also means that the verification of the code against the requirements is now trivial and completely automatic. The bottom line is another big jump in both productivity and first-time quality.
In summary, having a full set of automated tests is one of the most important things you can do, because tests:
Are bug-repellent
Help localize defects
Facilitate changes (act as a safety net)
Simplify integration
Improve code design (which also facilitates changes)
Document the actual behavior of the code
Get The Art of Lean Software Development now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
