Beautiful Testing

Conclusions

Change-centric testing is not a difficult task to accomplish. Once there is a good understanding of the extent of coverage provided by the existing test suites, the call graph dependencies in the executable, and the details of change made to the source code in a given interval, it will become a trivial task to generate a dynamic list of tests that test the change. Empirical experiments on large code bases have shown that the list of bugs that slipped detection is much shorter, and a lot of test cycle time is saved in this approach. This approach is beneficial when the existing “run all tests on a nightly basis” takes more than few hours to complete. With a change-centric testing approach, the size of the test suite that needs to be run on a nightly basis can be reduced dramatically.

Also, a change-centric testing approach provides a good insight into gaps in code coverage versus daily changes in the code base. This is a useful input for targeted test development.

However, this approach does not advocate that you stop running full-cycle system tests periodically. Full-cycle system tests need to be run at regular but longer intervals instead of daily or nightly intervals.