The Flaw of Averages: Why We Underestimate Risk in the Face of Uncertainty

CHAPTER 18

Simpson’s Paradox

Simpson’s Paradox is an insidious example of the Flaw of Averages that has tripped up sophisticated researchers for years. It is most easily understood in terms of a graph in which a straight line (known as a REGRESSION line) is fit to data so as to indicate the average change in one variable given a change in the other variable. Consider a fictitious dietary supplement based on rutabaga juice that appears to cause rapid weight loss. Figure 18.1 displays the results of 48 test subjects who have been on various doses of the supplement.

Figure 18.1 Weight change versus daily intake.

The x axis displays the daily intake of the stuff for each of the 48 subjects, and the y axis shows the pounds of weight gained or lost over the one-week test. The downward-sloping line indicates an incredible average of 1½ pounds of weight lost per week per gram of the supplement. This looks great so far, but before jumping to conclusions, we need to know more about the data.

Figure 18.2 Weight change versus daily intake by sex (male denoted by dark, female by light).

As it turns out, another variable was not accounted for in the data: the sex of the subjects. Figure 18.2 displays the data separated into males (dark dots) and females (light dots), whereupon it becomes obvious ...