O'Reilly logo

PRAGMATIC Security Metrics by W. Krag Brotby, Gary Hinson

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

75
Chapter 6
Metametrics and the
PRAGMATIC Approach
PRAGMATIC
security metrics
Chapter 1
Introduction
Chapter 13
Conclusion
Chapter 2
Why measure?
Chapter 12
Case study
Chapter 3
Art and science
Chapter 11
Using metrics
Chapter 4
Audiences
Chapter 10
Downsides
Chapter 5
Finding metrics
Chapter 9
Advanced metrics
Chapter 6
PRAGMATIC
Chapter 8
Measurement
system
Chapter 7
Example metrics
Appendices
Metametrics
Criteria
Scorin
g
Classifyin
g
A few well-chosen metrics can be a huge help in monitoring controls
and measuring their effectiveness
Clint Kreitner, Center for Internet Security
A few well-chosen metrics” sounds great! Trouble is, out of the vast range of infor-
mation security things that we might measure, which ones should we actually
select, analyze, and report? To put that another way, how do we distinguish good
from bad metrics? Which metrics have the qualities we desire? What are those
desirable qualities, criteria, or parameters, in fact?
76 ◾  PRAGMATIC Security Metrics
It is curious that so little has been said on what we feel is such an extremely
important topic. Sure, there are some rather academic reasons why the accountants
might prefer net present value over payback period when assessing the projected
value of security investments, but in practical terms and in some situations, pay-
back period may have redeeming qualities that make it the more valuable security
metric. Other security metrics books belabor the differences between ordinal and
cardinal numbers or metrics, measurements, and measures, but few information
security practitioners truly understand or even care much about such arcane details,
valid as they are. We simply need relevant, useful information in order to manage
and deliver adequate information security while our managers and other stakehold-
ers are clamoring for assurance that we have things under control.
e bottom line is that very few organizations today would claim to have effec-
tive and comprehensive information security metrics. Security metrics are arguably
the capstone for an information security management system,* the final piece of the
puzzle that locks all the others firmly in place.
Absent good information security metrics, we are flying blind.
6.1 Metametrics
We define metametrics,
like metadata, as information about metrics, including
metrics about metrics (e.g., “number of metrics supporting the information security
management system” qualifies as a metametric). Metametrics include descriptions
of metrics (e.g., most metrics catalogs consist of records for each metric containing
fields such as scope, purpose, parameters, sources, and calculations: these are all
metametrics). In exactly the same way that information security metrics are used to
measure, manage, and improve the security controls and, hence, the security man-
agement system, metametrics help us measure, manage, and improve our metrics
and, hence, the measurement system.
Metametrics are used
As indicators of the relative worth or value of each metric when considering
various options and choosing the best metrics for the measurement system
(see Chapter 8)
*
Even ISO/IEC JTC1/SC27, the international committee of highly experienced, competent,
and well-respected information security experts responsible for the ISO27k standards, strug-
gles with security metrics. Just look at ISO/IEC 27004 to see what we mean! It’s all very well
in theory, but how do we actually use it?
Stefani and Xenos (2009) used the term “metametric” in relation to choosing good eCom-
merce metrics, noting “Meta-metrics represent different aspects of the measurement procedure
like automation, measurement issues and reliability of provided measures.” We have simply
contracted the term.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required