Improving Surveys with Paradata: Analytic Uses of Process Information

CHAPTER 9

MANAGING DATA QUALITY INDICATORS WITH PARADATA BASED STATISTICAL QUALITY CONTROL TOOLS: THE KEYS TO SURVEY PERFORMANCE

MATT JANS

University of California, Los Angeles

ROBYN SIRKIS and DAVID MORGAN

U.S. Census Bureau

9.1 INTRODUCTION

Survey data and the paradata that describe how they were collected can be thought of as emerging from a system of interdependent processes, each of which includes individual moving parts. There is the recruitment process, during which units are sampled and approached for participation. From those approaches, case dispositions, like “Completed Survey,” or “Doorstep Refusal,” become paradata that describe this phase of the data collection system. There is the process of measuring the participating sample units, which produces paradata in the form of keystroke files and time stamps that yeild measures of question duration and interview pace. Recording, plotting, and monitoring the paradata produced by these operations offers opportunities to control these processes. Using paradata this way to manage survey data collection can lead to more efficient operations that produce higher quality data at lower cost (Groves and Heeringa, 2006). This chapter discusses opportunities and challenges for survey analytics, quality control, and quality assurance using paradata. It covers the role of statistical process and quality control concepts in paradata-based quality control and assurance programs, as well as an overview of key performance indicators (KPIs) ...