Introduction to Statistics Through Resampling Methods and R, 2nd Edition

Chapter 6

Designing an Experiment or Survey

Suppose you were a consulting statistician* and were given a data set to analyze. What is the first question you would ask? “What statistic should I use?” No, your first question always should be “How were these data collected?”

Experience teaches us that garbage in, garbage out or GIGO. In order to apply statistical methods, you need to be sure that samples have been drawn at random from the population(s) you want represented and are representative of those populations. You need to be sure that the observations are independent of one another and that outcomes have not been influenced by the actions of the investigator or survey taker.

Many times people who consult statisticians don’t know the details of the data collection process or they do know and look guilty and embarrassed when asked. All too often, you’ll find yourself throwing your hands in the air and saying, “if only you’d come to me to design your experiment in the first place.”

The purpose of this chapter is to take you step by step through the design of an experiment and a survey. You’ll learn the many ways in which an experiment can go wrong. And you’ll learn the right things to do to ensure your own efforts are successful.

6.1 THE HAWTHORNE EFFECT

The original objective of the industrial engineers at the Hawthorne plant of Western Electric was to see whether a few relatively inexpensive improvements would increase workers’ productivity. They painted the walls green and ...