O'Reilly logo

PRAGMATIC Security Metrics by W. Krag Brotby, Gary Hinson

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

v
Contents
Foreword ........................................................................................................xi
Preface ......................................................................................................... xiii
Acknowledgments .........................................................................................xv
Office Memorandum .................................................................................. xvii
1 Introduction ...........................................................................................1
1.1 Why Have We Written is Book?....................................................2
1.2 What’s Different about is Metrics Book? .......................................3
1.3 Who Are We Writing is For? .........................................................5
1.4 Who Are We? ....................................................................................5
1.4.1 W. Krag Brotby ....................................................................5
1.4.2 Gary Hinson ........................................................................7
1.5 What We’ll Be Talking about ............................................................8
1.6 Defining Our Terminology ...............................................................9
1.7 What We Expect of You, the Reader ...............................................10
1.8 Summary .........................................................................................11
2 Why Measure Information Security? ...................................................13
2.1 To Answer Awkward Management Questions ................................. 15
2.2 To Improve Information Security, Systematically ............................18
2.3 For Strategic, Tactical, and Operational Reasons .............................20
2.4 For Compliance and Assurance Purposes ........................................22
2.5 To Fill the Vacuum Caused by Our Inability to Measure Security ....... 23
2.6 To Support the Information Security Manager ................................24
2.7 For Profit! ........................................................................................25
2.8 For Various Other Reasons ..............................................................26
2.9 Summary ......................................................................................... 27
3 e Art and Science of Security Metrics ............................................... 29
3.1 Metrology, the Science of Measurement ..........................................30
3.2 Governance and Management Metrics ............................................30
vi ◾  Contents
3.3 Information Security Metrics ..........................................................32
3.4 Financial Metrics (for Information Security) ...................................33
3.5 (Information Security) Risk Management Metrics ..........................35
3.6 Software Quality (and Security) Metrics .........................................36
3.7 Information Security Metrics Reference Sources .............................37
3.7.1 Douglas Hubbard: How to Measure Anything
(Hubbard 2010) ..................................................................37
3.7.2 Andrew Jaquith: Security Metrics (Jaquith 2007) ................38
3.7.3 NIST SP 800-55: Performance Measurement Guide for
Information Security (NIST 2008) ......................................39
3.7.4 Debra Herrmann: Complete Guide to Security and
Privacy Metrics (Herrmann 2007) ...................................... 40
3.7.5 W. Krag Brotby: Information Security Management
Metrics (Brotby 2009a) .......................................................41
3.7.6 Lance Hayden: IT Security Metrics (Hayden 2010) .............41
3.7.7 Caroline Wong: Security Metrics: A Beginner’s Guide
(Wong 2012) ..................................................................... 42
3.7.8 ISO/IEC 27004: Information Security Management–
Measurement (ISO/IEC 27004 2009) ................................ 42
3.7.9 CIS Security Metrics (CIS 2010) ..........................................43
3.7.10 ISACA ............................................................................... 44
3.8 Specifying Metrics .......................................................................... 46
3.9 Metrics Catalogs and a Serious Warning about SMD ......................48
3.10 Other (Information Security) Metrics Resources .............................49
3.11 Summary ......................................................................................... 50
4 Audiences for Security Metrics .............................................................51
4.1 Metrics Audiences Within the Organization ....................................52
4.1.1 Senior Management ............................................................ 53
4.1.2 Middle and Junior Management .........................................54
4.1.3 Security Operations ............................................................55
4.1.4 Others with Interest in Information Security......................56
4.2 Metrics Audiences From Without the Organization ......................... 57
4.3 Summary ......................................................................................... 58
5 Finding Candidate Metrics ..................................................................59
5.1 Preexisting/Current Information Security Metrics ..........................60
5.2 Other Corporate Metrics ................................................................. 61
5.3 Metrics Used in Other Fields and Organizations ............................ 66
5.4 Information Security Metrics Reference Sources .............................67
5.5 Other Sources of Inspiration for Security Metrics ...........................68
5.5.1 Security Surveys .................................................................68
5.5.2 Vendor Reports and White Papers ......................................69
5.5.3 Security Software ...............................................................70
Contents ◾  vii
5.6 Roll-Your-Own Metrics ...................................................................70
5.7 Metrics Supply and Demand ...........................................................71
5.8 Summary ......................................................................................... 72
6 Metametrics and the PRAGMATIC Approach.....................................75
6.1 Metametrics ....................................................................................76
6.2 Selecting Information Security Metrics ...........................................78
6.3 PRAGMATIC Criteria .................................................................... 81
6.3.1 P = Predictive .....................................................................82
6.3.2 R = Relevant .......................................................................85
6.3.3 A = Actionable ....................................................................86
6.3.4 G = Genuine .......................................................................87
6.3.5 M = Meaningful .................................................................88
6.3.6 A = Accurate ...................................................................... 90
6.3.7 T = Timely .........................................................................91
6.3.8 I = Independent ..................................................................93
6.3.9 C = Cost .............................................................................94
6.4 Scoring Information Security Metrics against the
PRAGMATIC Criteria ....................................................................95
6.5 Other Uses for PRAGMATIC Metametrics ..................................104
6.6 Classifying Information Security Metrics ......................................105
6.6.1 Strategic/Managerial/Operational (SMO)
Metrics Classification .......................................................106
6.6.2 Risk/Control Metrics Classification....................................108
6.6.3 Input–ProcessOutput (Outcome) Metrics Classification ........ 108
6.6.4 Effectiveness and Efficiency Metrics Classification ..............109
6.6.5 Maturity Metrics Classification .........................................109
6.6.6 Directness Metrics Classification .......................................110
6.6.7 Robustness Metrics Classification .......................................110
6.6.8 Readiness Metrics Classification ........................................111
6.6.9 Policy/Practice Metrics Classification .................................112
6.7 Summary .......................................................................................113
7 150+ Example Security Metrics ..........................................................115
7.1 Information Security Risk Management Example Metrics ............118
7.2 Information Security Policy Example Metrics ...............................130
7.3 Security Governance, Management, and Organization
Example Metrics ............................................................................140
7.3.1 Information Security Financial Management Metrics ......141
7.3.2 Information Security Control-Related Metrics .................141
7.3.3 Metrics for Business Alignment and Relevance of
Controls............................................................................142
7.3.4 Control Monitoring and Testing Metrics ..........................143
7.3.5 Financial Information Security Metrics ............................ 156

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required