People trust numbers. They are the core of computation, the fundamentals of finance, and an essential part of human progress in the past century. And for the majority of modern societies, numbers are how we know things: we use them to study which drugs are safe, what policies work, and how our universe evolves. They are, perhaps, one of the most powerful and potent of human tools.
But like any tool, numbers aren’t good for everything. There are some kinds of questions for which numbers answer little. For example, public health researchers in the 1980s wanted to explain epileptic patients’ difficulties with taking their medication regularly. Researchers measured how many people failed to comply; they looked for statistical differences between those who complied and those who didn’t; they even ran elaborate longitudinal controlled experiments to measure the consequences of noncompliance. But none of these approaches explained the lack of compliance; they just described it in rigorous but shallow ways.
Then, a groundbreaking study [Conrad 1985] took a different approach. Instead of trying to measure compliance quantitatively, the researchers performed 80 interviews of people with epilepsy, focusing on the situations in which their informants were expected to take their medications but did not. The researchers found that the lack of compliance was due not to irrational, erratic behavior, but to patients’ deliberate choice. For example, some patients were aware of the potential of becoming chemically dependent on the drugs and carefully regulated their use of the drug to avoid tolerance. These qualitative findings, one of the first of their kind in public health research, became a critical part of redefining how epilepsy medication is delivered to society.
What does all of this have to do with software engineering? Like public health, software engineering is full of why and how questions for which numbers and statistics are of little help. For example, if you’ve managed a software team, you’ve probably asked yourself a number of questions. Why won’t my developers write unit tests? Why do users keep filling out this form incorrectly? Why are some of my developers 10 times as productive as the others? These questions can’t be answered with numbers, but they can with the careful application of qualitative methods.
But using qualitative methods isn’t as simple as asking people a few questions, nor is reading reports of qualitative studies as simple as reading the abstract. This chapter explains what qualitative methods are, how to interpret the results of qualitative studies, and how you might use them in your own work to improve software process and quality.
Get Making Software now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
