O'Reilly logo

Experiment!: Planning, Implementing and Interpreting by Oivind Andersson

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

8.2 The Importance of Randomization

If the lady should lack the ability to distinguish between the treatments, the experiment must be governed by chance for the significance test to be useful. As we mentioned in the last chapter, there are always numerous factors in experiments that are more or less outside our control, which may still affect the outcome. In this example, the quantity of milk might differ between the cups, the strength of the tea and the tasting temperature may change over time, and so forth. All of these factors could cause variations in taste that are greater than that caused by the experimental treatment. If, for instance, different persons were to prepare the cups belonging to the different treatments, systematic variations in taste could be introduced that were not due to the treatments. We could go to considerable lengths trying to eliminate such disturbances, inventing ever more accurate methods of pouring the milk and brewing the tea, but it would be a futile exercise to try to do away with them completely. It simply cannot be done.

It is important to keep in mind that experiments are always conducted using limited resources. When planning an experiment we must consider how to best utilize these resources. Which causes of disturbance should we care about and which ought to be deliberately ignored? How much effort should be put into diminishing the magnitude of those disturbances that we choose not to ignore? Most importantly, since it is impossible to eliminate ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required