The Fundamentals of Telemetry Pipelines

Chapter 3. Managing Your Telemetry Pipelines

Quis custodiet ipsos custodes?

(“Who will guard the guards themselves?” or “Who watches the watchers?”)

Juvenal

Telemetry pipelines establish control of your flood of telemetry data. Sources bring your telemetry data into the pipeline, destinations define where your data should surface, and processors transform, enrich, and augment that data in all the ways you need to turn the flood into a useful trickle.

In this chapter, you’re going to explore the different forms that your control can take, from the processors that are your building blocks to a real use case of pulling together a telemetry pipeline to condition and route a rich stream of data to your observability tools.

Managing and Debugging Your Pipelines

Developers, engineers, and other system builders are naturally optimists. We have to be; otherwise, we might be paralyzed by the ramifications of what we build if, or when, it goes wrong. But things do go wrong. Murphy’s law is real. That’s one of the reasons we want great telemetry underpinning powerful observability tools in the first place.

But Murphy’s law extends to your telemetry pipelines as well. As you take full advantage of the multifarious sources at your disposal and grapple with them using the many processors in your toolkit, things can and will go wrong, and those slips can be costly. This is why you need similar techniques when working with your telemetry pipelines as you do when working with your systems: great ...