Managing the Testing Process: Practical Tools and Techniques for Managing Hardware and Software Testing

1.3. What You Should Test: Considering Quality

Once I've identified the areas of testing that might be appropriate for my test organization, my next step is to figure out what I should test. To do this, I must understand what quality means for the system and the risks to system quality that exist. While quality is sometimes seen as a complex and contentious topic, I have found a pragmatic approach.

1.3.1. Three Blind Men and an Elephant: Can You Define Quality?

There's a management parable about three blind men who come across an elephant. One touched the tail and declared it a snake. Another touched a leg and insisted that it was a tree. The third touched the elephant's side and claimed that it was a wall.

Defining quality can be a similar process. Everyone "knows" what they mean, but disagreements abound. Have you debated with developers over whether a particular test case failure was really a bug? If so, weren't these debates in fact about whether the observed behavior was a quality issue? What, really, is quality? What factors determine its presence or absence? Whose opinions matter most?

J. M. Juran, a respected figure in the field of quality management, defines quality as "features [that] are decisive as to product performance and as to 'product satisfaction'.... The word 'quality' also refers to freedom from deficiencies... [that] result in complaints, claims, returns, rework and other damage. Those collectively are forms of 'product dissatisfaction.'"^[] Testing focuses ...