Monitoring Taxonomy

Chapter 10. traditional.free_open.processors.system

Tools in this category are breaking new ground, providing a general-purpose tool-chain for a problem that is commonly faced by data-centric organizations of any scale: how do we take metrics data from all of these various sources, and combine them to create a common telemetry signal?

Riemann: The Monitoring Leatherman

Riemann is an expressive and powerful stream-processing system for monitoring data. It ingests “events,” which are protobuf-encoded objects that represent state changes (OK, WARN, etc.), or metrics (foos:4, etc.). These events are then fed through a series of nested filters that can do all sorts of interesting things with them like enumarating them, joining them together, sending emails based on their content, forwarding them to visualization systems, and so on. (The sky is the limit.)

Many different clients and language bindings exist to help you transform whatever ad hoc monitoring data you have into Riemann events and emit them into Riemann. The configuration file literally is a Clojure program, so some familiarity with Clojure is recommended; however the documentation includes a primer that will have anyone who can program in any language up and running fairly quickly.

Riemann is a difficult piece of software to blithely sum up. It is conceptually simple, and yet basically impossible for a non-programming systems administrator to comprehend and use. I use it all the time and highly recommend it.

Push, Pull, ...