High-Performance TensorFlow in Production
Published by O'Reilly Media, Inc.
Develop hands-on experience optimizing and deploying Tensorflow models
This course will give you hands-on experience deploying an optimized Tensorflow Model into production for real-time prediction serving. You’ll build a lightweight Machine Learning/AI prediction service, similar to AWS and Google Cloud ML. You’ll train and export your Tensorflow Model using a Jupyter Notebook and the Python-based Tensorflow development libraries. And you’ll learn the structure of a Tensorflow Model, key components of Tensorflow Serving, including versioning and rollback, and how to optimize and simplify our trained Tensorflow Model through various techniques to reduce prediction latency.
You’ll also tune Tensorflow Serving to increase prediction throughput, and deploy your model with C++-based Tensorflow Serving to serve high-performance, real-time predictions.
While model training is part of this course, we focus mainly on model optimizing and serving. We may also cover parts of Distributed Tensorflow, Docker, and Kubernetes.
What you’ll learn and how you can apply it
You’ll understand:
- The structure of a Tensorflow Model
- Key components of Tensorflow Serving
- How to optimize a Tensorflow Model for serving
- How to tune Tensorflow Serving for performance
- How to deploy Tensorflow models with Tensorflow Serving
- How to version and rollback models with Tensorflow Serving
And you'll be able to:
- Optimize trained Tensorflow Models to reduce prediction latency
- Deploy trained Tensorflow Models to Tensorflow Serving in production
- Tune the Tensorflow Serving runtime to increase prediction throughput
- Version and roll-back models with Tensorflow Serving
This live event is for you because...
- You are a Software Engineer or Data Engineer with Intermediate Production-Deployment Experience and need to learn to deploy Tensorflow models to production.
- You are a Data Scientist or Business Analyst with intermediate ML or AI experience and need to learn to optimize Tensorflow models for production deployment.
Prerequisites
- Intermediate software engineering or data science skills.
Setup required prior to the first course meeting:
- The only requirement is a modern browser (ie. Chrome, Firefox, etc).
- Every attendee will get their own cloud instance accessible via their browser. The instructor will provide the IP addresses to each attendee at the beginning of the course. All work will be done using Jupyter notebooks running on each attendee’s assigned cloud instance.
- All work can be saved locally. The instructor will provide download instructions at the end of the course.
Schedule
The time frames are only estimates and may vary according to how the class is progressing.
Day 1: TensorFlow Model Training
- TensorFlow and GPUs
- Inspect and Debug Models
- Distributed Training Across a Cluster
- Optimize Training with Queues, Dataset API, and JIT XLA Compiler
Day 2: TensorFlow Model Deploying and Serving Predictions
- Optimize Predicting with AOT XLA and Graph Transform Tool (GTT)
- Key Components of TensorFlow Serving
- Optimize TensorFlow Serving Runtime
Your Instructor
Chris Fregly
Chris Fregly is a passionate performance engineer, AI product leader, and author of AI Systems Performance Engineering, Generative AI on AWS, and Data Science on AWS. He has worked at tech companies such as Netflix, Databricks, and AWS. Chris has led performance-focused engineering teams that built advanced AI products, scaled go-to-market initiatives, and reduced cost for large-scale generative AI and analytics workloads.