Observability Engineering

Chapter 7. Instrumentation with OpenTelemetry

In the previous two chapters, we described the principles of structured events and tracing. Events and traces are the building blocks of observability that you can use to understand the behavior of your software applications. You can generate those fundamental building blocks by adding instrumentation code into your application to emit telemetry data alongside each invocation. You can then route the emitted telemetry data to a backend data store, so that you can later analyze it to understand application health and help debug issues.

In this chapter, we’ll show you how to instrument your code to emit telemetry data. The approach you choose might depend on the instrumentation methods your observability backend supports. It is common for vendors to create proprietary APM, metrics, or tracing libraries to generate telemetry data for their specific solutions. However, for the purposes of this vendor-neutral book, we will describe how to implement instrumentation with open source standards that will work with a wide variety of backend telemetry stores.

This chapter starts by introducing the OpenTelemetry standard and its approach for automatically generating telemetry from applications. Telemetry from automatic instrumentation is a fine start, but the real power of observability comes from custom attributes that add context to help you debug how your intended business logic is actually working. We’ll show you how to use custom instrumentation ...