Making Software

How Evidence Turns Up in Software Engineering

In crime dramas, the most interesting moments are often when a new piece of evidence appears: someone presents a fact that was not previously known, and the clever detective revises his theory of the crime. Sometimes the new evidence adds an important dimension to our world view (“Oh, so he had intended all these things to happen!”), sometimes it just adds a little more conviction to a belief we already had (“She’s really not as careful as she claims”), and sometimes it overturns something we were sure was reliable (“Golly! And I always thought he had arrived there an hour after her!”). The drama hinges on the critical thinking ability of the detective. Superior fictional detectives take into account every single bit of evidence, they work until they can integrate it all into a coherent whole, and they continually test their theory of the crime and revise it in the light of new evidence, whereas weak detectives get snagged on confirmatory bias, holding fast to theories even when they can’t account for “loose ends” of evidence.

Software development is similar: evidence emerges over time, and the quality of the engineering hinges on the critical faculties of the engineer. If we succumb to confirmatory bias, we pay most attention to evidence that confirms our views. If we’re more critically vigilant, we sometimes find that new information suggests we have overlooked some aspect that has to be added to the picture, or we even find that some ...