How to Measure Anything in Cybersecurity Risk

Chapter 9 Some Powerful Methods Based on Bayes

If one fails to specify the prior information, a problem of inference is just as ill-posed as if one had failed to specify the data . . . In realistic problems of inference, it is typical that we have cogent prior information, highly relevant to the question being asked; to fail to take it into account is to commit the most obvious inconsistency of reasoning, and it may lead to absurd or dangerously misleading results.

—Edwin T. Jaynes

Recall that in our survey, 23% of respondents agreed with the statement “Probabilistic methods are impractical because probabilities need exact data to be computed and we don’t have exact data.” This is just a minority, but even those who rejected that claim probably have found themselves in situations where data seemed too sparse to make a useful inference. In fact that may be why the majority of the survey takers also responded that ordinal scales have a place in measuring uncertainty. Perhaps they feel comfortable using wildly inexact and arbitrary values like “high, medium, and low” to communicate risk while ironically still believing in quantitative approaches. Yet someone who thoroughly believed in using quantitative methods would roundly reject ordinal scales when measuring highly uncertain events. When you are highly uncertain you use probabilities and ranges to actively communicate your uncertainty—particularly when you are relying on subject matter expertise. Having read the earlier research ...