Improving Surveys with Paradata: Analytic Uses of Process Information

CHAPTER 14

THE QUALITY OF PARADATA: A LITERATURE REVIEW

BRADY T. WEST

Survey Research Center, Institute for Social Research, University of Michigan-Ann Arbor

JENNIFER SINIBALDI

Institute for Employment Research (IAB), Nuremberg

14.1 INTRODUCTION

When collecting paradata for the purpose of improving survey operations or data quality, the quality of the paradata must be understood. Comprehensive analyses from Carroll et al. (2006) and Fuller (1987) document the impact on final statistics when variables prone to measurement error are used in statistical modeling. The cautions from these findings extend to paradata, as inaccurate paradata could lead to biased nonresponse adjustments (Chapter 15; Biemer et al., 2013; West, 2013), increases (rather than decreases) in survey costs, and diminished quality of survey data. Studies examining the error properties of paradata are slowly beginning to emerge, but more are needed to justify the large quantities of paradata that survey organizations are collecting.

This chapter reviews the existing literature on the quality of paradata. We consider studies presenting both direct evaluations of the quality of a variety of collected paradata, where “gold standard” validation data are available, as well as indirect indicators of the quality, including reliability, ease of collection, and missing data issues. We first provide an overview of the available literature on computer-generated paradata and interviewer-recorded call records, much of which ...