Chapter 21
Evaluating Test Cases
Quis custodiet ipsos custodes?
Juvenal (Satire VI, lines 347–8)
ca. late 1st century, a.d.
Just as the Roman satirist Juvenal asked who would guard the guards, no matter how carefully a set of tests is developed, software testers should ask how good are their test cases. Edsger Dykstra observed that testing can detect the presence of faults but can never assert their absence. Who tests the test cases? More precisely, how can a set of test cases be tested? One answer has been around for more than 30 years—mutation testing. A more recent addition, “fuzzing,” is closer to random testing. The idea that fishing creel counts as an estimate of test case success is both novel and successful. We take a brief look at all ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access