Hands-On APIs for AI and Data Science

Chapter 13. Deploying a Machine Learning API

Always in motion is the future.

Yoda, The Empire Strikes Back

Fantasy football managers spend most of their time attempting to predict the future and plotting strategies based on those predictions. Before the season begins, managers want to know how NFL players will perform in the upcoming season so that they can build the best team. During their fantasy drafts, managers want to know where a player would be picked by other managers so that they can outmaneuver their competition. Each week, managers want to know which of their players are going to score the most so that they can set their lineups accordingly.

Many fantasy websites and platforms provide predictions to these managers. One of the tools available to the platforms is a machine learning (ML) model, which you learned about in Chapter 12. The platforms train various models and use them to make predictions, or inferences, to managers. If a model processes an entire group of predictions at once, it is called batch inference. Some fantasy questions are appropriate for batch inference, such as making a week’s worth of player predictions all at once. Batch inference may be done by a scheduled script or job. But if the predictions are changing minute by minute—like in the case of a live score prediction for a game—then real-time inference is needed. Real-time inference is calling a model to get a single prediction immediately. This is where deploying the model as an API is most ...