Total Survey Error in Practice

12Measurement Error in Survey Operations Management: Detection, Quantification, Visualization, and Reduction

Brad Edwards, Aaron Maitland, and Sue Connor

Westat, Rockville, MD, USA

Quantifying sampling error had become the sine qua non of survey methodology by the late 1930s, but by mid‐century statisticians and methodologists were taking note of many forms of nonsampling errors. The total survey error (TSE) paradigm brought into focus the various guises: coverage, validity, measurement, and nonresponse (Groves, 1989). Nonresponse is a concern if the nonresponders are different from responders in important ways, and thus nonresponse error can cause bias. The nonresponse bias can in some cases be estimated, but attempts to quantify all types of nonsampling errors in a comprehensive error measure for a survey have been relatively rare. Efforts to employ TSE in designing or managing surveys have been marked by uncertainty and guesswork. In survey operations, managers seek to balance efforts to reduce measurement error and nonresponse error within cost constraints, and paradata tools can assist. As probability surveys face increasing cost pressure and response rate challenges, managers could benefit from more rigorous measures of survey error as they seek to make informed tradeoffs between quality and costs. Making well‐informed tradeoffs to reduce error while the data are still being collected is often better than estimating (and living with) the error afterward. That is the ...