Common Errors in Statistics (and How to Avoid Them), 4th Edition

HIGHER-ORDER EXPERIMENTAL DESIGNS

The two principal weaknesses of the analysis of variance are as follows:

1. The various tests of significance are not independent of one another as they are based on statistics that share a common denominator;

2. Undefined confounding variables may create the illusion of a relationship or may mask an existing one.

When we randomly assign subjects (or plots) to treatment, we may inadvertently assign all males, say, to one of the treatments. The result might be the illusion of a treatment effect that really arises from a sex effect. For example, the following table

suggests there exists a statistically significant difference between treatments.

But suppose, we were to analyze the same data correcting for sex and obtain the following:

We longer observe a statistically significant difference between treatment groups.

Errors in Interpretation

As noted previously, one of the most common statistical errors is to assume that because an effect is not statistically significant it does not exist. One of the most common errors in using the analysis of variance is to assume that because a factor such as sex does not yield a significant p-value that we may eliminate it from the model. Had we done so in the above example, we would have observed a statistically ...