Serving Machine Learning Models

Model serving is a critical but often underappreciated aspect of machine learning.Once you have built a model using your training data set, you need to packageand deploy (i.e., serve) it. It's a surprisingly complex task, in part because modeltraining is usually handled by data scientists, and model serving is the domain ofsoftware engineers. These two groups have different functions, concerns, andtools, so the handoff can be tricky. Plus, machine learning is a hot and fast-growing field, spawning a slew of new tools that require software engineers tocreate new model serving frameworks.

This book delves into the theory and practice of serving machine learning modelsin streaming applications. It proposes an overall architecture that implementscontrolled streams of both data and models that enables not only real-time modelserving, as part of processing input streams, but also real-time model updating. Italso covers:

• Step-by- step options for exporting models in tensorflow and PMMLformats.
• Implementation of model serving leveraging stream processing enginesand frameworks including Apache Flink, Apache Spark streaming, ApacheBeam, Apache Kafka streams, and Akka streams.
• Monitoring approaches for model serving implementations.