Observability Engineering, 2nd Edition

Chapter 8. Getting Started with Observability Analysis

In Part II of this book, you learned about the fundamentals of instrumenting your code to emit telemetry. But that’s only half of the equation. Collecting the right data is a fundamental requirement, but the true measure of observability is what you can learn about your systems from that data.

This chapter looks at how gathering the telemetry data you learned about radically changes how you’re able to understand the behavior of your code. Working in high-observability systems, the analysis techniques you’ll use differ drastically from those you typically use in more traditional, low-observability systems.

We’ll start by examining the analysis techniques used in low-observability systems. Those approaches heavily rely on prior system familiarity and experience for investigators to successfully find answers. In contrast, with high-observability systems, investigators start with a structured process—the core analysis loop—that leads them to the correct source of issues. That difference results in both leveling the playing field when identifying issues and quickly accelerating the fixing of them.

The core analysis loop can be accomplished manually, but it’s best when automated. In looking at how to automate analysis, we’ll consider the roles that both humans and computers play in creating effective debugging workflows. We’ll provide guidance when using AI assistants to automate telemetry data analysis. By demonstrating these analysis ...