Practical Machine Learning for Computer Vision

Chapter 9. Model Predictions

The primary purpose of training machine learning models is to be able to use them to make predictions. In this chapter, we will take a deep dive into several considerations and design choices involved in deploying trained ML models and using them to make predictions.

Tip

The code for this chapter is in the 09_deploying folder of the book’s GitHub repository. We will provide file names for code samples and notebooks where applicable.

Making Predictions

To invoke a trained model—i.e., to use it to make predictions—we have to load the model from the directory into which it was exported and call the serving signature. In this section, we will look at how to do this. We will also look at how to improve the maintainability and performance of invoked models.

Exporting the Model

To obtain a serving signature to invoke, we must export our trained model. Let’s quickly recap these two topics—exporting and model signatures—which were covered in much greater detail in “Saving Model State” in Chapter 7. Recall that a Keras model can be exported (see the notebook 07c_export.ipynb on GitHub) using code like this:

model.save('gs://practical-ml-vision-book/flowers_5_trained')

This saves the model in TensorFlow SavedModel format. We discussed examining the signature of the prediction function using the command-line tool saved_model_cli. By default the signature matches the input layer of the Keras model that was saved, but it is possible to export the model with ...