Willful Ignorance: The Mismeasure of Uncertainty

CHAPTER 1 THE OPPOSITE OF CERTAINTY

During the past century, research in the medical, social, and economic sciences has led to major improvements in longevity and living conditions. Statistical methods grounded in the mathematics of probability have played a major role in much of this progress. Our confidence in these quantitative tools has grown, along with our ability to wield them with great proficiency. We have an enormous investment of tangible and intellectual capital in scientific research that is predicated on this framework. We assume that the statistical methods as applied in the past so successfully will continue to be productive. Yet, something is amiss.

New findings often contradict previously accepted theories. Faith in the ability of science to provide reliable answers is being steadily eroded, as expert opinion on many critical issues flip-flops. Scientists in some fields seriously debate whether a majority of their published research findings are ultimately overturned¹; the decline effect has been coined to describe how even strongly positive results often fade over time in the light of subsequent study²; revelations of errors in the findings published in prestigious scientific journals, and even fraud, are becoming more common.³ Instead of achieving greater certainty, we seem to be moving backwards. What is going on?

Consider efforts to help disadvantaged children through early childhood educational intervention. Beginning around 1970, the U.S. government ...