What Is MLOps?

Chapter 3. Model Monitoring and Iteration

Since ML models are effectively models of the data they were trained on, they can degrade over time. This is not a problem faced by traditional software, but it is inherent to machine learning. ML mathematics builds a concise representation of the important patterns in the training data with the hope that this is a good reflection of the real world. If the training data reflects the real world well, then the model should be accurate and, thus, useful.

But the real world doesn’t stand still. The training data used to build a fraud detection model six months ago won’t reflect a new type of fraud that started to occur in the last three months. If a given website starts to attract an increasingly younger user base, then a model that generates advertisements is likely to produce less and less relevant ads.

Once a model is in use, it is crucial that it continues to perform well over time. But good performance means different things to different people, in particular to data scientists and to the business. This chapter will take a closer look at the monitoring and iteration steps of the AI project life cycle and the role the business plays in the utility of both processes.

Why Model Monitoring Matters

Model moderating and iteration is the bread and butter of MLOps. And when it comes to monitoring the performance of models and of AI projects, it’s important to recognize that everyone in the room (in other words, everyone involved with the ...