Common Errors in Statistics (and How to Avoid Them), 4th Edition

EXPERIMENTAL DESIGN

Statisticians have found three ways for coping with individual-to-individual and observer-to-observer variation:

1. Controlling. The fewer the extrinsic sources of variation, the smaller the sample size required. Make the environment for the study—the subjects, the manner in which the treatment is administered, the manner in which the observations are obtained, the apparatus used to make the measurements, and the criteria for interpretation—as uniform and homogeneous as possible.

2. Blocking. A clinician might stratify the population into subgroups based on such factors as age, sex, race, and the severity of the condition, and restrict comparisons to individuals who belong to the same subgroup. An agronomist would want to stratify on the basis of soil composition and environment.Blocking can also be performed after the experiment for the purpose of analysis but only if you have taken the time to record the blocking variable.

3. Randomizing. Randomly assign patients to treatment within each subgroup so that the innumerable factors that can neither be controlled nor observed directly are as likely to influence the outcome of one treatment as another.

Steps one and two are trickier than they appear at first glance. Do the phenomena under investigation depend upon the time of day, as with body temperature and the incidence of mitosis? Upon the day of the week, as with retail sales and the daily mail? Will the observations be affected by the sex of the observer? ...