Statistics for Data Science and Analytics

7Surveys and Sampling

In this chapter, we discuss sampling as a device to gain a more accurate picture of your data. After you complete this chapter, you should be able to:

• Define statistical bias
• Explain the key feature of an ideal simple random sample (SRS)
• Use resampling to derive sampling distributions for proportions and means
• Calculate confidence intervals
• Define the bootstrap
• Describe sampling methods other than simple random sampling

7.1 Literary Digest—Sampling Trumps “All Data”

By the end of 1936, the United States was showing signs of economic recovery from the Great Depression that started with the 1929 collapse of Wall Street. GDP was back to where it was in 1929; it had fallen by a third in the interim. Unemployment was headed back to 15% after having risen to 25% at the depths of the recession. The good news was destined to be short-lived. Another recession hit in 1937, and the economy stagnated until World War II. However, in 1936, things were looking up for President Roosevelt as he campaigned for a second term. He had successfully put Social Security and Unemployment Compensation legislation through Congress and was banking on the popularity of his New Deal platform.

Political polling was in its infancy, though, so the preferences of the electorate remained somewhat obscure. In each presidential election year since 1916, The Literary Digest, a national weekly opinion magazine that often featured Norman Rockwell illustrations on its cover, had mailed out ...