How to Measure Anything in Cybersecurity Risk

Chapter 4 The Single Most Important Measurement in Cybersecurity

We hope Chapter 2 cleared up how the term “measurement” is used in decision science as well as the empirical sciences in general. We contend that this is the most relevant understanding of measurement for cybersecurity. Chapter 3 gave you an introduction to the most basic level of quantitative risk analysis. There will be a lot more to cover regarding the details of the methods of measurement, but for now we propose that our first target should be a measurement of risk analysis itself.

The authors have observed experts throughout the field with passionately held positions on the relative merits of different cybersecurity risk-assessment methods. One easy observation we could make is that both sides of polar-opposite positions were often argued by highly qualified cybersecurity experts, all with decades of experience. One knowledgeable expert will argue, for example, that a particular framework based on qualitative scores improves decisions, builds consensus, and avoids the problems of more quantitative methods. Another equally qualified expert will argue this is an illusion and that such methods simply “do the math wrong.” Since we know at least one (if not both) must be wrong, then we know qualifications and expertise in cybersecurity alone are not sufficient to determine if a given opinion on this topic is correct.

This leaves us with several hard questions. How do we decide which methods work better? Is it ...