Taking Testing Seriously

APPENDIX CHeuristics of Testability

The practical testability of a product is how easy it is to test¹ by a particular tester and test process, in a given context.² Practical testability is a function of five other “testabilities:” project-related testability, value-related testability, subjective testability, intrinsic testability, and epistemic testability (also known as the risk gap). Just as in the case of quality in general, testability is a plastic and multidimensional concept that cannot be usefully expressed in any single metric. But we can identify testability problems and heuristics for improving testability in general.

Interesting Testability Dynamics

• Changing the product or raising the quality standard reduces epistemic testability. The difference between what we know and what we need to know is why we test in the first place. A product is easier to test if we already know a lot about its quality or if the quality standard is low, because there isn't much left for testing to do. That's epistemic testability. Therefore, a product that changes a lot or one in which we can't tolerate trouble is automatically less testable.
• Improving any other aspect of testability increases the rate of improvement of epistemic testability. Any effort made to improve any other aspect of testability, by definition, increases the rate at which the gap between what we know and what we ...