Chaos Engineering

Chapter 16. Continuous Verification

Continuous verification is a discipline of proactive experimentation in software, implemented as tooling that verifies system behaviors.

Casey Rosenthal

The challenges posed by complex systems encouraged a natural evolution from continuous integration to continuous delivery to continuous verification. The latter is the subject of this chapter, describing the nascent field and the room for opportunity, and following up with a real-world example of one of the systems in production at Netflix called ChAP. Where the industry decides to take continuous verification is wide open, but the general areas of focus for future development are reviewed at the end of this chapter.

Where CV Comes From

When a gap of expectations exists between two or more engineers separately writing code, then the interaction of that code may produce unexpected and undesirable results. The quicker that can be caught, the more likely it is that the next code written will have fewer gaps. Conversely, if that gap is not caught early on, then the next code written will likely diverge even further, multiplying the opportunities for undesirable outcomes.

One of the most efficient methods for uncovering that expectations gap is to put the code together and run it. Continuous integration (CI) was promoted heavily as part of the XP methodology as a way to achieve this. CI is now a common industry norm. CI pipelines encourage the development of integration tests, which specifically ...