Common Errors in Statistics (and How to Avoid Them), 4th Edition

Chapter 1

Sources of Error

Don’t think—use the computer.

Dyke (tongue in cheek) [1997].

We cannot help remarking that it is very surprising that research in an area that depends so heavily on statistical methods has not been carried out in close collaboration with professional statisticians, the panel remarked in its conclusions. From the report of an independent panel looking into “Climategate.”¹

STATISTICAL PROCEDURES FOR HYPOTHESIS TESTING, ESTIMATION, AND MODEL building are only a part of the decision-making process. They should never be quoted as the sole basis for making a decision (yes, even those procedures that are based on a solid deductive mathematical foundation). As philosophers have known for centuries, extrapolation from a sample or samples to a larger, incompletely examined population must entail a leap of faith.

The sources of error in applying statistical procedures are legion and include all of the following:

1. 

a) Replying on erroneous reports to help formulate hypotheses (see Chapter 9)

b) Failing to express qualitative hypotheses in quantitative form (see Chapter 2)

c) Using the same set of data both to formulate hypotheses and to test them (see Chapter 2)

2. 

a) Taking samples from the wrong population or failing to specify in advance the population(s) about which inferences are to be made (see Chapter 3)

b) Failing to draw samples that are random and representative (see Chapter 3)

3. Measuring the wrong variables or failing to measure what you intended ...