How to Measure Anything: Finding the Value of Intangibles in Business, 3rd Edition

CHAPTER 5 Calibrated Estimates: How Much Do You Know Now?

The most important questions of life are indeed, for the most part, really only problems of probability.

—Pierre Simon Laplace, Théorie Analytique des Probabilités, 1812

How many hours per week do employees spend addressing customer complaints? How much would sales increase with a new advertising campaign? Even if you don’t know the exact values to questions like these, you still know something. You know that some values would be impossible or at least highly unlikely. Knowing what you know now about something actually has an important and often surprising impact on how you should measure it or even whether you should measure it. In fact, quantifying our current level of uncertainty is a key part of the statistical methods we are using and the second step in the universal measurement approach—just after defining the decision and just before computing the value of additional information. What we need is a way to express how much we know now, however little that may be. To do that, we need a way to know if we are any good at expressing uncertainty.

One method to express our uncertainty about a number is to think of it as a range of probable values. In statistics, a range that has a particular chance of containing the correct answer is called a confidence interval (CI). A 90% CI is a range that has a 90% chance of containing the correct answer. For example, you can’t know for certain exactly how many of your current prospects ...