Video description
TensorFlow Serving is one of the cornerstones in the TensorFlow ecosystem. It has eased the deployment of machine learning models tremendously and led to an acceleration of model deployments. Unfortunately, machine learning engineers aren’t familiar with the details of TensorFlow Serving, and they’re missing out on significant performance increases.
Hannes Hapke (SAP ConcurLabs) provides a brief introduction to TensorFlow Serving, then leads a deep dive into advanced settings and use cases. He introduces advanced concepts and implementation suggestions to increase the performance of the TensorFlow Serving setup, which includes an introduction to how clients can request model meta-information from the model server, an overview of model optimization options for optimal prediction throughput, an introduction to batching requests to improve the throughput performance, an example implementation to support model A/B testing, and an overview of monitoring your TensorFlow Serving setup.
Prerequisite knowledge
- A basic understanding of Docker functionality and how HTTP requests work
- General knowledge of machine learning (useful but not required)
What you'll learn
- Learn how to increase the TensorFlow Serving inference performance, increase the inference response time by tweaking the request payload, and run TensorFlow Serving with NVIDIA's TensorRT for further performance improvements
- Discover how to configure batch requests in TensorFlow Serving and how to configure TensorFlow Serving to provide A/B Testing capabilities
This session is from the 2019 O'Reilly TensorFlow World Conference in Santa Clara, CA.
Product information
- Title: Advanced model deployments with TensorFlow Serving
- Author(s):
- Release date: February 2020
- Publisher(s): O'Reilly Media, Inc.
- ISBN: 0636920372585
You might also like
video
Deploying TensorFlow Models to a Web Application: Using Flask API, TensorFlowJS, and TensorFlow Serving
Implement machine learning to realize the power of AI algorithms. Developers and companies often struggle to …
video
Hands-On Transfer Learning with TensorFlow 2.0
Transfer learning involves using a pre-trained model on a new problem. It is currently very popular …
book
Learn TensorFlow Enterprise
Use TensorFlow Enterprise with other GCP services to improve the speed and efficiency of machine learning …
video
TFX: Production ML pipelines with TensorFlow
ML development often focuses on metrics, delaying work on deployment and scaling issues. ML development designed …