Managing the Testing Process: Practical Tools and Techniques for Managing Hardware and Software Testing

11.5. Case Study

Table 11.1 shows the return on investment for three test projects. The first column is the type of project. Test development includes all costs associated with developing test cases, data, scripts, and other reusable intellectual property in the test system. Test environment includes systems, tools, and other reusable real property in the test lab. Both test development and test environment are amortized over three years. (The laptop computer project did not include any test development or environment because we used two outside test labs for our effort, which allowed us to capture this amortization as built-in overhead on test execution.)

Test execution includes the effort spent running the tests, gathering the results, reporting the results, responding to changes, and so forth; this is not amortized as it applies to the project itself. "Bugs" is the number of bugs found in testing. I calculate return on investment assuming an internal cost of failure associated with the must-fix bugs of $100 versus an external cost of failure of $1,000. This is also an estimate because precise cost of quality accounting data was not available. (See exercise 2 for a demonstration of why this is a conservative assumption.) Therefore, the return on investment is calculated as:

Figure 11.7. The testing process.

Retrospectively, I have self-assessed the maturity of the test process ...