The Flaw of Averages: Why We Underestimate Risk in the Face of Uncertainty

PART 9

TOWARD A CURE FOR THE FLAW OF AVERAGES

The story that I have to tell is marked all the way through by a persistent tension between those who assert that the best decisions are based on quantification and numbers, determined by the patterns of the past, and those who base their decisions on more subjective degrees of belief about the uncertain future. This is a controversy that has never been resolved.

—Peter L. Bernstein, Against the Gods: the Remarkable Story of Risk

 

 

The point of view based on “patterns of the past” is classical statistics, somewhat analogous to classical music. It is historical and based on rigid and elaborate rules. Although mathematically aesthetic (I cannot imagine life without Bach), it cannot be expected to keep up with the quickly changing rhythms of our uncertain world. Classical statistics shines at determining the average lengths of swan beaks to three decimal places given thousands of white swans. But it will never predict a black one from purely white data.

The form of reasoning based on “subjective degrees of belief about the uncertain future” is like improvised jazz, in which the musicians commit to their own notes in advance of knowing with certainty what the others are going to play. If there were a field of statistics to deal with black swans, it would be this improvisational form.

The instruments played by the improvisational statistician are based on interactive visualization. Those wishing to join in will not wait for classical training ...