How to Measure Anything in Cybersecurity Risk, 2nd Edition

CHAPTER 7Calibrated Estimates: How Much Do You Know Now?

The most important questions of life are indeed, for the most part, really only problems of probability.

—Pierre‐Simon Laplace, Théorie Analytiquedes Probabilités, 1812¹

The method described so far requires the subjective evaluation of quantitative probabilities. For example, the cybersecurity expert will need to assess a probability that an event will occur or how much will be lost if it does. This meets some resistance. Some cybersecurity experts, many of whom seem to have no issue with assigning a “medium” or a “2” to a likelihood, will often wonder how it is possible to subjectively assess a quantitative probability of an event. Of course, it is legitimate to ask whether subjective probabilities can be valid. Fortunately, as mentioned in Chapter 5, much research has already been done on this point, and two findings are clear: (1) Most people are bad at assigning probabilities, but (2) most people can be trained to be very good at it.

Yes, the validity of subjective estimates of probability can be and has been objectively measured (ironically, perhaps to some). To deny this is a rejection of scientifically validated facts. A cybersecurity expert can learn how to express their uncertainty with a subjective—but quantitative—expression of uncertainty. In this chapter we will introduce the basic idea of using subjective estimates of probabilities. We will also show how your skill at doing this can be measured and improved ...