Improving Surveys with Paradata: Analytic Uses of Process Information

CHAPTER 4

ANALYZING PARADATA TO INVESTIGATE MEASUREMENT ERROR

TING YAN

Institute for Social Research, University of Michigan; formerly NORC at the University of Chicago

KRISTEN OLSON

University of Nebraska-Lincoln

4.1 INTRODUCTION

Paradata for purposes of investigating and understanding measurement error include response times, keystrokes, mouse clicks, behavior codes, vocal characteristics, and interviewer evaluations. Description and collection of these paradata is examined extensively in Chapter 3. In this chapter, we focus on the analysis of these types of paradata.

We begin this chapter with a brief review of empirical literature using paradata to investigate measurement error in Section 4.2. Section 4.3 describes the specific analytic steps taken and issues to be considered when analyzing paradata for the purpose of examining measurement error. Examples taken from published research work are provided in Section 4.4 to illustrate how one type of paradata—response times—has been used to answer various research questions on measurement error. Section 4.5 calls the reader’s attention to issues related to the measurement error in paradata themselves and other general issues involved in analyzing paradata. We wrap up this chapter by offering, in Section 4.6, take home points for researchers, survey practitioners, supervisors, and interviewers.

4.2 REVIEW OF EMPIRICAL LITERATURE ON THE USE OF PARADATA FOR MEASUREMENT ERROR INVESTIGATION

Measurement error refers to the difference ...