Chaos Engineering

Chapter 1. Encountering Complex Systems

In the first part of this chapter we explore the problems that arise when dealing with complex systems. Chaos Engineering was born of necessity in a complex distributed software system. It specifically addresses the needs of operating a complex system, namely that these systems are nonlinear, which makes them unpredictable, and in turn leads to undesirable outcomes. This is often uncomfortable for us as engineers, because we like to think that we can plan our way around uncertainty. We are often tempted to blame these undesirable behaviors on the people who build and operate the systems, but in fact surprises are a natural property of complex systems. Later in this chapter we then ask whether we can extricate the complexity out of the system, and in doing so extricate the undesirable behaviors with it. (Spoiler: no, we cannot.)

Contemplating Complexity

Before you can decide whether Chaos Engineering makes sense for your system, you need to understand where to draw the line between simple and complex. One way to characterize a system is in the way changes to input to the system correspond to changes in output. Simple systems are often described as linear. A change to input of a linear system produces a corresponding change to the output of the system. Many natural phenomena constitute familiar linear systems. The harder you throw a ball, the further it goes.

Nonlinear systems have output that varies wildly based on changes to the constituent ...