The Failure of Risk Management, 2nd Edition

CHAPTER 7The Limits of Expert Knowledge: Why We Don't Know What We Think We Know about Uncertainty

Experience is inevitable. Learning is not.

—PAUL J. H. SCHOEMAKER

We are riding the early waves of a 25-year run of a greatly expanding economy that will do much to solve seemingly intractable problems like poverty and to ease tensions throughout the world.

—WIRED (JULY 1997)

Chapter 3 explained the need for component testing in methods for risk management and risk assessment. One critical component we need to test would be the human experts we must rely on for so much of the input. In fact, the simple method introduced in chapter 4—similar to the risk matrix it replaces—will rely heavily on the human expert input. (Empirical methods to improve on expert input will be introduced later.)

Experts may assess a probability quantitatively, as we would do with the chapter 4 model (e.g., “There is a 10 percent chance this project will fail”). If, however, the expert is using a risk matrix, he or she may express this chance on some sort of scale other than explicit probabilities (e.g., “On a scale of 1 to 5, the likelihood of this project failing is a 2”). A risk matrix method may also do without any numbers altogether—ordinal or otherwise—and provide this estimate in some purely verbal form (e.g., “It is unlikely this project will fail”). And whether the risk assessment is lacking any quantities or using advanced quantitative methods, human beings must use their judgment to identify ...