The Art of Software Testing, Second Edition

7.7. Error Analysis

The last thing to realize about program debugging is that, in addition to its value in removing an error from the program, it can have another valuable effect: It can tell us something about the nature of software errors, something we still know too little about. Information about the nature of software errors can provide valuable feedback in terms of improving future design, coding, and testing processes.

Every programmer and programming organization could improve immensely by performing a detailed analysis of the detected errors, or at least a subset of them. It is a difficult and time-consuming task, for it implies much more than a superficial grouping such as "x percent of the errors are logic-design errors," or "x percent of the errors occur in IF statements." A careful analysis might include the following studies:

• Where was the error made? This question is the most difficult one to answer, because it requires a backward search through the documentation and history of the project, but it also is the most valuable question. It requires that you pinpoint the original source and time of the error. For example, the original source of the error might be an ambiguous statement in a specification, a correction to a prior error, or a misunderstanding of an end-user requirement.

• Who made the error? Wouldn't it be useful to discover that 60 percent of the design errors were created by one of the 10 analysts, or that programmer X makes three times as many mistakes ...