Building an ML/AI Pipeline with Kubeflow and MLflow
Published by O'Reilly Media, Inc.
Continuous machine learning and AI
Kubeflow and MLflow are open source projects dedicated to end-to-end machine learning using the latest AI best practices, including hyperparameter tuning, AutoML, and experiment tracking, to find the best algorithms and models to fit your dataset.
Join experts Chris Fregly and Antje Barth to learn how to build a real-world machine learning pipeline using Kubeflow, MLflow, TensorFlow, Keras, and Apache Spark in a Kubernetes environment. Along the way, you’ll explore model deploying, A/B testing, and multiarmed bandits—tools to help you quickly deploy your models into production with zero downtime.
What you’ll learn and how you can apply it
By the end of this live online course, you’ll understand:
- The core components of the Kubeflow ecosystem
- Modern best practices for ML/AI model training, tuning, and deploying
- How to build your own Kubeflow pipeline for your machine learning use cases
- How to deploy a Jupyter notebook to a Kubernetes cluster
- How to use hyperparameter tuning and AutoML to find the best models
- How to use experiment tracking to visually compare model performance
- How to safely deploy models directly to production with zero downtime
- How to compare models in production using A/B testing and multiarmed bandits
And you’ll be able to:
- Set up continuous machine learning pipelines with Kubeflow and MLflow
- Train models using Jupyter, TensorFlow, Keras, PyTorch, and Apache Spark
- Perform hyperparameter tuning to find the best model
- Compare models using experiment tracking
- Deploy models directly to production with Kubeflow Serving and Istio
- Use A/B testing and multiarmed bandits with production models
This live event is for you because...
- You’re a data scientist or ML engineer who needs to develop AI/ML pipelines for production applications.
- You’re a data engineer who needs to manage AI/ML pipelines in production.
- You’re a data analyst or business intelligence specialist who wants to understand the landscape of end-to-end AI/ML pipelines.
Prerequisites
- A computer with a modern web browser installed (Chrome is recommended, but Firefox is OK.)
Schedule
The time frames are only estimates and may vary according to how the class is progressing.
Introduction (20 minutes)
- Presentation: Exploring the environment
- Hands-on exercise: Set up and explore the environment
- Q&A
Kubeflow overview (20 minutes)
- Presentation: Kubeflow Pipelines use cases
- Hands-on exercise: Create a Kubeflow pipeline with TensorFlow
- Q&A
MLflow overview (15 minutes)
- Presentation: MLflow pipelines use cases
- Hands-on exercise: Create an MLflow pipeline with Apache Spark and Keras
- Q&A
Break (5 minutes)
Hyperparameter tuning (15 minutes)
- Presentation: Hyperparameter tuning use cases
- Hands-on exercise: Perform hyperparameter tuning
- Q&A
AutoML (15 minutes)
- Presentation: AutoML use cases
- Hands-on exercise: Implement AutoML
- Q&A
Experiment tracking (15 minutes)
- Presentation: Experiment tracking with Kubeflow and MLflow use cases
- Hands-on exercise: Track and compare experiments
- Q&A
Break (15 minutes)
Model deploying (15 minutes)
- Presentation: Model deploying options
- Hands-on exercise: Deploy two versions of a model using Kubeflow Serving and Istio
A/B testing (15 minutes)
- Presentation: A/B testing use cases
- Hands-on exercise: Perform an A/B test with the two versions of the model
Multiarmed bandit testing (15 minutes)
- Presentation: Multiarmed bandit testing use cases
- Hands-on exercise: Perform a multiarmed bandit test with the two versions of the model
Wrap-up and Q&A (15 minutes)
Your Instructors
Chris Fregly
Chris Fregly is a passionate performance engineer, AI product leader, and author of AI Systems Performance Engineering, Generative AI on AWS, and Data Science on AWS. He has worked at tech companies such as Netflix, Databricks, and AWS. Chris has led performance-focused engineering teams that built advanced AI products, scaled go-to-market initiatives, and reduced cost for large-scale generative AI and analytics workloads.
Antje Barth
Antje Barth is a principal developer advocate for generative AI at Amazon Web Services. She’s also coauthor of the O’Reilly books Generative AI on AWS and Data Science on AWS. A frequent speaker at AI and machine learning conferences and meetups around the world, she cofounded the global Generative AI on AWS Meetup and the Düsseldorf chapter of Women in Big Data. Previously, Antje worked in solutions engineering roles at MapR and Cisco, helping developers leverage big data, containers, and Kubernetes platforms in the context of AI and machine learning.