Improving Surveys with Paradata: Analytic Uses of Process Information

CHAPTER 3

COLLECTING PARADATA FOR MEASUREMENT ERROR EVALUATIONS

KRISTEN OLSON and BRYAN PARKHURST

University of Nebraskas-Lincoln

3.1 INTRODUCTION

Survey researchers and methodologists seek to have new and innovative ways of evaluating the quality of data collected from sample surveys. Paradata, or data collected for free from computerized survey instruments, have increasingly been used in survey methodological work for this purpose (Couper, 1998). One error source that has been studied using paradata is measurement error, or the deviation of a response from a “true” value (Groves, 1989; Biemer and Lyberg, 2003). Although used in psychological literature since the 1980s (see Fazio, 1990, for an early review) and adapted to telephone interviews by Bassili in the early 1990s (Bassili and Fletcher, 1991; Bassili and Scott, 1996), the adoption and use of paradata for studying measurement-error-related outcomes has grown exponentially with the growth of web surveys and increased use of computerization in interviewer-administered surveys (Couper, 1998; Heerwegh, 2003; Couper and Lyberg, 2005). Paradata are a proxy for breakdowns in the cognitive response process or identify problems respondents and interviewers have with a survey instrument (Couper, 2000; Yan and Tourangeau, 2008).

Paradata can be collected at a variety of levels, resulting in a complex, hierarchical data structure. Examples of paradata collected automatically by many computerized survey software systems include timing ...