Designing Machine Learning Systems

Designing Machine Learning Systems

by Chip Huyen
Released May 2022
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781098107963

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Buy on Amazon Buy on ebooks.com
Start your free trial

Book description

Machine learning systems are both complex and unique. Complex because they consist of many different components and involve many different stakeholders. Unique because they're data dependent, with data varying wildly from one use case to the next. In this book, you'll learn a holistic approach to designing ML systems that are reliable, scalable, maintainable, and adaptive to changing environments and business requirements.

Author Chip Huyen, co-founder of Claypot AI, considers each design decision--such as how to process and create training data, which features to use, how often to retrain models, and what to monitor--in the context of how it can help your system as a whole achieve its objectives. The iterative framework in this book uses actual case studies backed by ample references.

This book will help you tackle scenarios such as:

  • Engineering data and choosing the right metrics to solve a business problem
  • Automating the process for continually developing, evaluating, deploying, and updating models
  • Developing a monitoring system to quickly detect and address issues your models might encounter in production
  • Architecting an ML platform that serves across use cases
  • Developing responsible ML systems

Publisher resources

View/Submit Errata

Table of contents

  1. Preface
    1. Who This Book Is For
    2. What This Book Is Not
    3. Navigating This Book
    4. GitHub Repository and Community
    5. Conventions Used in This Book
    6. Using Code Examples
    7. O’Reilly Online Learning
    8. How to Contact Us
    9. Acknowledgments
  2. 1. Overview of Machine Learning Systems
    1. When to Use Machine Learning
      1. Machine Learning Use Cases
    2. Understanding Machine Learning Systems
      1. Machine Learning in Research Versus in Production
      2. Machine Learning Systems Versus Traditional Software
    3. Summary
  3. 2. Introduction to Machine Learning Systems Design
    1. Business and ML Objectives
    2. Requirements for ML Systems
      1. Reliability
      2. Scalability
      3. Maintainability
      4. Adaptability
    3. Iterative Process
    4. Framing ML Problems
      1. Types of ML Tasks
      2. Objective Functions
    5. Mind Versus Data
    6. Summary
  4. 3. Data Engineering Fundamentals
    1. Data Sources
    2. Data Formats
      1. JSON
      2. Row-Major Versus Column-Major Format
      3. Text Versus Binary Format
    3. Data Models
      1. Relational Model
      2. NoSQL
      3. Structured Versus Unstructured Data
    4. Data Storage Engines and Processing
      1. Transactional and Analytical Processing
      2. ETL: Extract, Transform, and Load
    5. Modes of Dataflow
      1. Data Passing Through Databases
      2. Data Passing Through Services
      3. Data Passing Through Real-Time Transport
    6. Batch Processing Versus Stream Processing
    7. Summary
  5. 4. Training Data
    1. Sampling
      1. Nonprobability Sampling
      2. Simple Random Sampling
      3. Stratified Sampling
      4. Weighted Sampling
      5. Reservoir Sampling
      6. Importance Sampling
    2. Labeling
      1. Hand Labels
      2. Natural Labels
      3. Handling the Lack of Labels
    3. Class Imbalance
      1. Challenges of Class Imbalance
      2. Handling Class Imbalance
    4. Data Augmentation
      1. Simple Label-Preserving Transformations
      2. Perturbation
      3. Data Synthesis
    5. Summary
  6. 5. Feature Engineering
    1. Learned Features Versus Engineered Features
    2. Common Feature Engineering Operations
      1. Handling Missing Values
      2. Scaling
      3. Discretization
      4. Encoding Categorical Features
      5. Feature Crossing
      6. Discrete and Continuous Positional Embeddings
    3. Data Leakage
      1. Common Causes for Data Leakage
      2. Detecting Data Leakage
    4. Engineering Good Features
      1. Feature Importance
      2. Feature Generalization
    5. Summary
  7. 6. Model Development and Offline Evaluation
    1. Model Development and Training
      1. Evaluating ML Models
      2. Ensembles
      3. Experiment Tracking and Versioning
      4. Distributed Training
      5. AutoML
    2. Model Offline Evaluation
      1. Baselines
      2. Evaluation Methods
    3. Summary
  8. 7. Model Deployment and Prediction Service
    1. Machine Learning Deployment Myths
      1. Myth 1: You Only Deploy One or Two ML Models at a Time
      2. Myth 2: If We Don’t Do Anything, Model Performance Remains the Same
      3. Myth 3: You Won’t Need to Update Your Models as Much
      4. Myth 4: Most ML Engineers Don’t Need to Worry About Scale
    2. Batch Prediction Versus Online Prediction
      1. From Batch Prediction to Online Prediction
      2. Unifying Batch Pipeline and Streaming Pipeline
    3. Model Compression
      1. Low-Rank Factorization
      2. Knowledge Distillation
      3. Pruning
      4. Quantization
    4. ML on the Cloud and on the Edge
      1. Compiling and Optimizing Models for Edge Devices
      2. ML in Browsers
    5. Summary
  9. 8. Data Distribution Shifts and Monitoring
    1. Causes of ML System Failures
      1. Software System Failures
      2. ML-Specific Failures
    2. Data Distribution Shifts
      1. Types of Data Distribution Shifts
      2. General Data Distribution Shifts
      3. Detecting Data Distribution Shifts
      4. Addressing Data Distribution Shifts
    3. Monitoring and Observability
      1. ML-Specific Metrics
      2. Monitoring Toolbox
      3. Observability
    4. Summary
  10. 9. Continual Learning and Test in Production
    1. Continual Learning
      1. Stateless Retraining Versus Stateful Training
      2. Why Continual Learning?
      3. Continual Learning Challenges
      4. Four Stages of Continual Learning
      5. How Often to Update Your Models
    2. Test in Production
      1. Shadow Deployment
      2. A/B Testing
      3. Canary Release
      4. Interleaving Experiments
      5. Bandits
    3. Summary
  11. 10. Infrastructure and Tooling for MLOps
    1. Storage and Compute
      1. Public Cloud Versus Private Data Centers
    2. Development Environment
      1. Dev Environment Setup
      2. Standardizing Dev Environments
      3. From Dev to Prod: Containers
    3. Resource Management
      1. Cron, Schedulers, and Orchestrators
      2. Data Science Workflow Management
    4. ML Platform
      1. Model Deployment
      2. Model Store
      3. Feature Store
    5. Build Versus Buy
    6. Summary
  12. 11. The Human Side of Machine Learning
    1. User Experience
      1. Ensuring User Experience Consistency
      2. Combatting “Mostly Correct” Predictions
      3. Smooth Failing
    2. Team Structure
      1. Cross-functional Teams Collaboration
      2. End-to-End Data Scientists
    3. Responsible AI
      1. Irresponsible AI: Case Studies
      2. A Framework for Responsible AI
    4. Summary
  13. Epilogue
  14. Index
  15. About the Author

Product information

  • Title: Designing Machine Learning Systems
  • Author(s): Chip Huyen
  • Release date: May 2022
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781098107963