Making Software

Chapter 4. Understanding Software Engineering Through Qualitative Methods

Andrew Ko

People trust numbers. They are the core of computation, the fundamentals of finance, and an essential part of human progress in the past century. And for the majority of modern societies, numbers are how we know things: we use them to study which drugs are safe, what policies work, and how our universe evolves. They are, perhaps, one of the most powerful and potent of human tools.

But like any tool, numbers aren’t good for everything. There are some kinds of questions for which numbers answer little. For example, public health researchers in the 1980s wanted to explain epileptic patients’ difficulties with taking their medication regularly. Researchers measured how many people failed to comply; they looked for statistical differences between those who complied and those who didn’t; they even ran elaborate longitudinal controlled experiments to measure the consequences of noncompliance. But none of these approaches explained the lack of compliance; they just described it in rigorous but shallow ways.

Then, a groundbreaking study [Conrad 1985] took a different approach. Instead of trying to measure compliance quantitatively, the researchers performed 80 interviews of people with epilepsy, focusing on the situations in which their informants were expected to take their medications but did not. The researchers found that the lack of compliance was due not to irrational, erratic behavior, but to patients’ ...