97 Things Every Programmer Should Know

Chapter 81. Test Precisely and Concretely

Kevlin Henney

IT IS IMPORTANT TO TEST for the desired, essential behavior of a unit of code, rather than for the incidental behavior of its particular implementation. But this should not be taken or mistaken as an excuse for vague tests. Tests need to be both accurate and precise.

Something of a tried, tested, and testing classic, sorting routines offer an illustrative example. Implementing a sorting algorithm is not necessarily an everyday task for a programmer, but sorting is such a familiar idea that most people believe they know what to expect from it. This casual familiarity, however, can make it harder to see past certain assumptions.

When programmers are asked, “What would you test for?”, by far and away the most common response is something like, “The result of sorting is a sorted sequence of elements.” While this is true, it is not the whole truth. When prompted for a more precise condition, many programmers add that the resulting sequence should be the same length as the original. Although correct, this is still not enough. For example, given the following sequence:

3 1 4 1 5 9

The following sequence satisfies a postcondition of being sorted in non-descending order and having the same length as the original sequence:

3 3 3 3 3 3

Although it satisfies the spec, it is also most certainly not what was meant! This example is based on an error ...