O'Reilly logo

PRAGMATIC Security Metrics by W. Krag Brotby, Gary Hinson

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

267
Chapter 9
Advanced Information
Security Metrics
PRAGMATIC
security metrics
Chapter 1
Introduction
Chapter 13
Conclusion
Chapter 2
Why measure?
Chapter 12
Case study
Chapter 3
Art and science
Chapter 11
Using metrics
Chapter 4
Audiences
Chapter 10
Downsides
Chapter 5
Finding metrics
Chapter 9
Advanced metrics
Chapter 6
PRAGMATIC
Chapter 8
Measurement
system
Chapter 7
Example metrics
Appendices
High-reliability
Indicators
KGIs, KRIs, CSFs
Targets, triggers
We are saddled with a culture that hasn’t advanced as far as science.
Michelangelo Antonioni
We dont mean to imply that the metrics practices we have discussed previously are
retarded as such, rather that there are even more sophisticated considerations than
we have so far considered. Many of the metrics issues discussed below have their
roots in well-established disciplines, such as commerce/business management, sci-
ence, and engineering. Compared to information security, or more accurately, IT
268 ◾  PRAGMATIC Security Metrics
security,* they are highly mature, tracing their histories back literally thousands
of years rather than mere decades. As information security professionals with an
interest in metrics, we have a lot to learn from our learned colleagues in other
disciplines.
9.1 High-Reliability Metrics
Metrics, like other processes, tools, and controls, sometimes fail. Unreliable instru-
ments or measurement processes are annoying at best, misleading us with inac-
curate, imprecise, or sporadic readings, implying that something is under control
when, in fact, it is not or failing to alert us to conditions that require our atten-
tion. At worst, they can be a liability, occasionally creating grave risks and cata-
strophic consequences.
Consequently, every metric of any importance
should be
considered in terms of whether, when, and how it might fail and ideally engineered
to make failure either extremely unlikely or conspicuously obvious. is section
concerns the application of fail-safe and related reliability engineering concepts to
information security metrics.
Safety-critical systems are the classic example. Many machines must operate
within certain ranges for safety reasons: operating parameters exceeding accept-
able limits would constitute a safety hazard, jeopardizing life and limb. e asso-
ciated measurements are not only used to operate/manage the machines but also
to confirm that they remain within safe limits, and hence, just like the machines
themselves, the measurements must be more than just ordinarily reliable. Ideally,
safety-critical machines and the associated measures and processes should fail safe,
for example, if a nuclear reactor core temperature exceeds a limit value (indicating
a control failure) or if the temperature readings dont make sense or stop altogether
for some reason (indicating a measurement failure), the control rods are dropped
automatically into the core to dampen the reaction. Approaches like these have
developed over many decades of industrial design, applied engineering, and trial
and error, learning from accidents, incidents, and near misses, and the learning pro-
cess continues every day. We are presently behind the curve in information security.
Broadly similar principles can be applied to the design of business-critical
processes, systems, controls, and measurements. High-reliability metrics areor
rather should bean integral part of that mix.
A few information security controls arguably fall into the safety-critical cat-
egory where the consequences of security failures are extreme hazards that threaten
*
e Caesar cypher, for instance, is about 2000 years old. Hail Caesar!
Speaking as someone who once ran out of fuel on an isolated stretch of road in the depths of
winter because of the car’s fuel gauge icing up and sticking at part-full when the tank was, in
fact, empty, I have a healthy respect for the reliability of measurements and instruments.
…and metrics of no importance are about as much use as ashtrays on motorbikes.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required