Making Software

Study Methodology

We conducted a direct observation case study and analyzed the data using grounded theory. Direct observation involves one of the researchers sitting in the same office as the participant, watching everything that the participant does and writing it down in a log. We often prefer direct observation to interviews or surveys because, as nonparticipant observers, we can be impartial and record every detail of the action, which gives us an exact view of events as they occurred. This contrasts with an interview or a survey, where subjects suffer from several biases. Typical are generalization bias, where they tell us generally how their day goes instead of giving us specific events, and memory bias, where they remember more recent events better than older ones. In qualitative studies like ours, the goal is not to eliminate all bias (which is impossible), but to choose the biases that will have the least influence on the results we uncover.

The case study aspect of our approach was to choose a small number of participants to observe. Participants were selected to be as different from one another as possible, so that we might observe a diverse set of activities, behaviors, and phenomena. Direct observation is an expensive way to gather data due to the time commitment of following someone around for many hours of the day. This limits the number of people we can include in the study, so we try to make sure that we see as much phenomena as we can from a small number of participants. ...