97 Things Every SRE Should Know

Chapter 95. Beyond Local Risk: Accounting for Angry Birds

Blake Bisset

Microsoft Azure

Data-driven analysis can only provide the kind of engineering investment guidance we want when applied to problems we already know we have—or should know, anyway. They’re already in our risk registry: things we’ve seen before and likely have seen relatively frequently if we actually associate a solid, quantifiable history of realized risk with them.

In dealing with known but unquantifiable risks, or the actual black swans and unknown unknowns, our best efforts often fail us, and we quickly reach a place where we need help in the divine or psychotherapeutic sense. Or possibly both.

These things will almost never turn up in our data until it’s too late, or if they do, they will still not be amenable to actual calculations of risk and predicted impact and cost over time. Although all these things are critical tools in the life of an engineer, they’re not inherently reliable, or at least not very likely to produce the kind of evidence that will convince a plurality of VPs to impose a Code Yellow.

This is where things get interesting, however. Over the course of building taxonomies of failure at a few companies, I’ve become very interested in the idea of expanding this mechanism beyond a single division or organization. What might we be able to achieve with a broader shared taxonomy and data pool? ...