O'Reilly logo

Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition by Aurélien Géron

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Chapter 19. Training and Deploying TensorFlow Models at Scale

Once you have a beautiful model that makes amazing predictions, what do you do with it? Well, you need to put it in production! This could be as simple as running the model on a batch of data, and perhaps writing a script that runs every night to do just that. However, it is often much more involved. Various parts of your infrastructure may need to use this model on live data, in which case you probably want to wrap your model in a webservice, so it can be queried at any time using a simple REST API (or some other protocol), as we discussed in Chapter 2. But as time passes, you need to retrain your model regularly on fresh data, and push the updated version to production. You must handle model versioning, gracefully transition from one model to the next, possibly rollback to the previous model in case of problems, and perhaps run multiple different models in parallel to perform A/B experiments.1 If your product becomes successful, your service may start to get plenty of queries per second (QPS), and it must scale up to support the load. A great solution is to use TF Serving, either on your own hardware infrastructure, or via a cloud service such as Google Cloud AI Platform, as we will see in this chapter. It will take care of efficiently serving your model, handle graceful model transitions, and more. If you use the cloud platform, you will also get many extra features, such as powerful monitoring tools.

Moreover, if ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required