book

Machine Learning Platform Engineering

Name: Machine Learning Platform Engineering
ISBN: 9781633437333

by Benjamin Tan Wei Hao, Varun Mallya, Shanoop Padmanabhan

February 2026

Intermediate to advanced

504 pages

14h 32m

English

Manning Publications

Read now

Unlock full access

Machine Learning Platform Engineering
copyright
contents
dedication
preface
acknowledgments
about this book
about the authors
about the cover illustration
Part 1 Laying the MLOps foundation

1 Getting started with MLOps and ML engineering
1.1 The ML life cycle1.1.1 Experimentation phase1.1.2 Development/staging/production phase1.2 Skills needed for MLOps1.2.1 Required skills for ML engineers1.2.2 Prerequisites1.3 Building an ML platform1.3.1 Build vs. buy1.3.2 Looking ahead: From MLOps to LLMOps1.3.3 Tools used in this book1.4 Building ML systems1.4.1 Introducing the ML projects1.4.2 ML projects
2 What is MLOps?
2.1 The iterative MLOps life cycle2.1.1 Data collection2.1.2 Exploratory Data Analysis2.1.3 Modeling and training2.1.4 Model evaluation2.1.5 Deployment2.1.6 Monitoring2.1.7 Maintenance, updates, and review2.2 Why is robust MLOps important ?2.3 Role of MLOps in a mature organization2.4 DevOps vs. MLOps2.5 Levels of MLOps maturity2.5.1 Level 0: Basic2.5.2 Level 1: Intermediate2.5.3 Level 2: Advanced
3 Building applications on Kubernetes
3.1 Containers and tooling3.2 Docker3.2.1 Write the application code3.2.2 Write a Dockerfile3.2.3 Building and pushing a Docker image3.3 Kubernetes3.3.1 Kubernetes architecture overview3.3.2 Kubectl3.3.3 Kubernetes objects3.3.4 Networking and services3.3.5 Other objects3.3.6 Helm charts3.3.7 Conclusion3.4 Continuous integration and deployment3.4.1 GitLab CI3.4.2 Argo CD3.5 Prometheus and Grafana
Part 2 Building core ML platform capabilities
4 Designing reliable ML systems
4.1 MLflow for experiment tracking4.1.1 Data exploration4.1.2 MLflow tracking4.1.3 MLflow model registry4.2 Feast as a feature store4.2.1 Registering features4.2.2 Retrieving features4.2.3 Feature server4.2.4 Using the Feast UI
5 Orchestrating ML pipelines
5.1 Kubeflow Pipelines: Task orchestrator5.1.1 Kubeflow components5.1.2 Income classifier pipeline
6 Productionizing ML models
6.1 BentoML as a deployment platform6.1.1 Building a Bento6.1.2 Building and pushing the Bento6.1.3 Deploying a Bento6.2 Evidently for data drift monitoring6.2.1 Data drift detection report and dashboard6.2.2 Data drift detection Kubeflow pipeline component6.2.3 Data drift detection for a model deployed as an API
Part 3 Applying MLOps in practice
7 Data analysis and preparation
7.1 Data analysis7.1.1 Launching a notebook server in Kubeflow7.1.2 Workspace and data volumes7.1.3 Configurations and affinity/tolerations7.1.4 Customizing the menu7.1.5 Creating a custom Kubeflow notebook image7.2 Data passing7.2.1 Scenario 1: Passing simple values to downstream components7.2.2 Scenario 2: Passing paths for larger data7.2.3 Overview of KFP v2 artifact types7.3 Data preparation in action7.3.1 Data preparation: Object detection7.3.2 Data preparation: Movie recommender
8 Model training and validation: Part 1
8.1 Training an object detection model8.1.1 Training YOLO on a custom dataset8.1.2 Training the model8.1.3 Container components for system dependencies8.1.4 Creating the validation component8.1.5 Creating the pipeline8.1.6 Executing the pipeline8.1.7 Validating model artifacts
9 Model training and validation: Part 2
9.1 Storing data with PersistentVolumeClaim9.1.1 Refactoring the pipeline with a PVC9.1.2 Efficient dataset management9.1.3 Creating a VolumeOp9.1.4 Download Op using PVC9.1.5 Splitting the dataset directly9.1.6 Simplifying model training9.1.7 Simplifying model validation9.2 Tracking training with TensorBoard9.2.1 Launching a new TensorBoard9.2.2 Exploring YOLOv8’s default graphs9.3 Movie recommender project9.3.1 Reading data from MinIO and quality assurance9.3.2 Model training component9.3.3 Metrics for evaluation9.3.4 Experiment tracking with MLflow9.3.5 Model registry with MLflow9.3.6 Creating a pipeline from components9.3.7 Local inference in a notebook
10 Model inference and serving
10.1 Model deployment is hard10.2 BentoML: Simplifying model deployment10.3 A whirlwind tour of BentoML10.3.1 BentoML Service and Runners10.4 Executing a BentoML Service locally10.4.1 Loading a model with BentoML Runner10.5 Building Bentos: Packaging your service for deployment10.5.1 Bento tags: Versioning and managing your Bentos10.6 BentoML and MLflow inference10.7 Using only MLflow to create an inference service10.8 KServe: An alternative to BentoML
11 Monitoring and explainability
11.1 Monitoring11.1.1 Basic monitoring11.1.2 Custom metrics11.1.3 Logging11.1.4 Alerting11.2 Data drift detection11.2.1 Object detection11.2.2 Movie recommender11.3 Explainability11.3.1 Object detection11.3.2 Movie recommendation
Part 4 Extending MLOps for large language models
12 Designing LLM-powered systems
12.1 LLMOps: New challenges, familiar principles12.1.1 What makes LLM applications different12.1.2 Extending our ML platform for LLMs12.1.3 Essential tools for LLM applications12.2 Building DataKrypt’s DakkaBot: A simple RAG architecture12.2.1 What you’ll build12.2.2 Beyond single API calls: Designing for composability12.2.3 Google’s Gemini LLM and embeddings12.2.4 The retrieval component12.2.5 The augmentation component12.2.6 The generation component12.3 Giving DakkaBot a UI12.4 Observability for LLM applications12.4.1 Set up Langfuse via Docker12.4.2 Integrating Langfuse with DakkaBot12.4.3 Enhanced observability in DakkaBotCore12.4.4 Beyond traditional metrics
13 Production LLM system design
13.1 Prompt engineering: Code for the generative AI era13.1.1 Treating prompts as critical infrastructure13.1.2 Langfuse prompt management for DakkaBot13.1.3 Langfuse prompt management for production13.2 Testing LLM applications13.2.1 Evaluation framework for LLM responses13.2.2 Safety and adversarial testing13.3 Governance and safety in production13.3.1 Implementing safety guardrails13.4 Cost optimization strategies13.4.1 Understanding LLM economics13.4.2 Model selection strategy13.4.3 Caching strategies13.4.4 Prompt optimization for efficiency13.4.5 Production cost monitoring13.4.6 From traditional ML to LLMOps
appendix A Installation and setup
A.1 Local installation of command-line tools (Mac and Linux)A.1.1 The yq YAML processorA.1.2 KustomizeA.1.3 KubectlA.1.4 K8s distributionA.1.5 K3s installationA.1.6 MicroK8s installationA.1.7 Argo CDA.1.8 KubeflowA.1.9 Cloud provider K8s setupA.1.10 MLflow setupA.2 Deploy MLflowA.2.1 Redis online store setupA.2.2 BentoML and Yatai setupA.2.3 Evidently UI setup
appendix B Basics of YAML
B.1 Basic YAML filesB.1.1 CommentsB.1.2 Scalar valuesB.1.3 ListsB.1.4 Nested structures (maps)B.1.5 Quoted stringsB.1.6 Multiline stringsB.1.7 Data types in YAMLB.2 Aliases and anchorsB.2.1 References (merging and reusing data)B.2.2 Complex data typesB.2.3 Custom data typesB.2.4 Block style vs. flow styleB.2.5 Key sorting and case sensitivityB.2.6 Best practices

Overview

Get your machine learning models out of the lab and into production!

Delivering a successful machine learning project is hard. Machine Learning Platform Engineering makes it easier. In it, you’ll design a reliable ML system from the ground up, incorporating MLOps and DevOps along with a stack of proven infrastructure tools including Kubeflow, MLFlow, BentoML, Evidently, and Feast.

In Machine Learning Platform Engineering you’ll learn how to:

Set up an MLOps platform
Deploy machine learning models to production
Build end-to-end data pipelines
Effective monitoring and explainability

A properly designed machine learning system streamlines data workflows, improves collaboration between data and operations teams, and provides much-needed structure for both training and deployment. In Machine Learning Platform Engineering you’ll learn how to design and implement a machine learning system from the ground up. You’ll appreciate this instantly-useful introduction to achieving the full benefits of automated ML infrastructure.

About the Technology
AI and ML systems have a lot of moving parts, from language libraries and application frameworks, to workflow and deployment infrastructure, to LLMs and other advanced models. A well-designed internal development platform (IDP) gives developers a defined set of tools and guidelines that accelerate the dev process, improving consistency, security, and developer experience.

About the Book
Machine Learning Platform Engineering shows you how to build an effective IDP for ML and AI applications. Each chapter illuminates a vital part of the ML workflow, including setting up orchestration pipelines, selecting models, allocating resources for training, inference, and serving, and more. As you go, you’ll create a versatile modern platform using open source tools like Kubeflow, MLFlow, BentoML, Evidently, Feast, and LangChain.

What's Inside

Set up an end-to-end MLOps/LLMOps platform
Deploy ML and AI models to production
Effective monitoring, evaluation, and explainability

About the Reader
For data scientists or software engineers. Examples in Python.

About the Authors
Benjamin Tan Wei Hao leads a team of ML engineers and data scientists at DKatalis. Shanoop Padmanabhan is a software engineering manager at Continental Automotive. Varun Mallya is a senior ML engineer at DKatalis.

Quotes
A great resource, especially for those looking for a hands-on approach.
- Noah Flynn, Amazon

Covers all the patterns you should follow.
- Andrew R. Freed, IBM

Rich and well structured.
- Vinicios Wentz, Nubank

Packed with code examples and capstone projects.
- Nupur Baghel, Google

A must-have if you want to learn how to build and deploy ML models from scratch.
- Ravikumar Sanapala, Meta

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781633437333Publisher Support Other Publisher Website Purchase Link

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills