Total Survey Error in Practice

5The Undercoverage–Nonresponse Tradeoff

Stephanie Eckman¹ and Frauke Kreuter^2,3,4

¹ Survey Research Division, RTI International, Washington, DC, USA

² Joint Program in Survey Methodology, University of Maryland, College Park, MD, USA

³ Department of Sociology, University of Mannheim, Mannheim, Germany

⁴ Statistical Methods Group, Institute for Employment Research (IAB), Nuremberg, Germany

5.1 Introduction

A survey that aims to provide estimates of means, proportions, or totals of a target population should represent the target population well, which in an ideal world means that all members of the target population have a known and positive probability to be selected for the survey (coverage) and that all selected and eligible cases take part in the survey (response). Surveys that exclude a portion of the target population from selection (e.g., those without phones) suffer from undercoverage. Surveys that fail to collect data from all eligible selected cases suffer from nonresponse. Although nonresponse and undercoverage are two distinct error sources in the total survey error (TSE) framework, in practice, we often see a connection between the two.

As a motivating example, consider a telephone survey of persons of retirement age. The interviewer calls a sampled phone number and asks the man who answers the phone if he is above 65 years of age. Suppose he is above 65, but he does not want to participate in the survey: he might find it easier to say that he is not eligible (not ...