Experiment!: Planning, Implementing and Interpreting

8.1 A Teatime Experiment

This example is borrowed from Fisher's classical book [1], as it presents some important ideas in an everyday context that requires no previous knowledge. Once you understand this simple example you will also understand why statistics is useful in the planning and interpretation of many experiments.

Imagine a lady who likes to take milk with her tea and claims that she can taste whether the milk was added to the cup before or after the tea. Our aim is to design an experiment to test this ability. We explain to her that we are going to prepare eight cups of tea. The milk will be added first to four of them, and after the tea to the other four. These two ways of preparing the tea will be called two experimental treatments. We will then present the cups to her in random order and ask her to find the four where the milk was added first.

Even if she does find the right cups, this could of course be a lucky coincidence. It is therefore useful to think about the all the possible outcomes and decide how to interpret them. Since her task is to choose four cups to put in one category, leaving the remaining four in the other, this particular experiment has 70 possible outcomes. This is because the first cup can be chosen in eight different ways, the second in seven ways and so on, yielding 8 × 7 × 6 × 5 = 1680 possible sets. However, many of these sets will contain the same cups, only arranged in different order. There are 4 × 3 × 2 × 1 = 24 ways of arranging a set ...