The Metrics Manifesto

CHAPTER 11Simulating Data Like a Pro

 

Note: This is a code heavy chapter. That's why I pushed it to later in the book. Yet, you can read at least half of this chapter without having any desire to code. If you are one of those readers, I do think it is important for you to push ahead and read. Starting with simulation is an important metrics design concept, one that is novel to security – but old news to the sciences.

Introduction

You can measure anything, but you can't measure everything. Why? Because measurement takes time. It takes skills. And, it takes money. If this wasn't true, more people would be doing it. That is why you need to prioritize what you are going to measure. Simulation can help.

Simulation forces you to think deeply about the object and methods of measurement. The object of measurement is the security process you want to improve. The methods are the metrics (algorithms) you will use.

Simulation helps you create rough but useful samples of the types of data you will ultimately consume. Those samples let you test out your algorithms (your metrics), removing a lot of the guesswork out of what might otherwise be an expensive and potentially fruitless effort.

The good news is, with simulation you don't need to wait for data.

Don't be fooled. Simulation is more than just spitting out data. It is modeling random data-generating processes. That means accounting ...