Serving Machine Learning Models

Chapter 9. Monitoring

General approaches for obtaining execution information include the following:

• Logging

• Writing this information (periodically) to an external storage (for example databases)

Although both of these approaches work and are often used, they typically suffer from the following:

• They introduce additional latency to the execution—they are typically implemented using synchronous calls within the execution itself.

• They require additional software components—database, log aggregators, and so on—which translates to additional maintenance

Both Flink and Kafka Streams recently introduced queryable state (Figure 9-1), which is a different approach to such monitoring.

The Kafka Streams documentation defines queryable state (interactive queries) as an approach, which, according to the documentation

lets you get more from streaming than just the processing of data. This feature allows you to treat the stream processing layer as a lightweight embedded database and, more concretely, to directly query the latest state of your stream processing application, without needing to materialize that state to external databases or external storage first.

Figure 9-1. Queryable state

Flink

Flink recently introduced a managed keyed-state interface that provides access to a state that is scoped to the key of the current execution. This means that this type of state can be ...