Machine Learning Design Patterns

Book Description

The design patterns in this book capture best practices and solutions to recurring problems in machine learning. Authors Valliappa Lakshmanan, Sara Robinson, and Michael Munn catalog the first tried-and-proven methods to help engineers tackle problems that frequently crop up during the ML process. These design patterns codify the experience of hundreds of experts into advice you can easily follow.

The authors, three Google Cloud engineers, describe 30 patterns for data and problem representation, operationalization, repeatability, reproducibility, flexibility, explainability, and fairness. Each pattern includes a description of the problem, a variety of potential solutions, and recommendations for choosing the most appropriate remedy for your situation.

You’ll learn how to:

  • Identify and mitigate common challenges when training, evaluating, and deploying ML models
  • Represent data for different ML model types, including embeddings, feature crosses, and more
  • Choose the right model type for specific problems
  • Build a robust training loop that uses checkpoints, distribution strategy, and hyperparameter tuning
  • Deploy scalable ML systems that you can retrain and update to reflect new data
  • Interpret model predictions for stakeholders and ensure that models are treating users fairly

Publisher Resources

View/Submit Errata

Table of Contents

  1. 1. The Need for Machine Learning Design Patterns
    1. What Are Design Patterns?
    2. Machine Learning Terminology
      1. Models and Frameworks
      2. Data and Feature Engineering
      3. The Machine Learning Process
      4. Data and Model Tooling
      5. Roles
    3. Common Challenges in Machine Learning
      1. Data Quality
      2. Reproducibility
      3. Data Drift
      4. Scale
      5. Multiple Objectives
    4. Machine Learning Systems
      1. ML Lifecycle
      2. AI Readiness
  2. 2. Data Representation Design Patterns
    1. Simple Data Representations
      1. Numerical Inputs
      2. Categorical Inputs
    2. Design Pattern 1: Hashed Feature
      1. Problem
      2. Solution
      3. Why It Works
      4. Tradeoffs and Alternatives
    3. Design Pattern 2: Reframing
      1. Problem
      2. Solution
      3. Why It Works
      4. Tradeoffs and Alternatives
    4. Design Pattern 3: Feature Cross
      1. Problem
      2. Solution
      3. Why It Works
      4. Tradeoffs and Alternatives
    5. Design Pattern 4: Multimodal Input Representations
      1. Problems
      2. Solution
      3. Tradeoffs and Alternatives
    6. Summary
  3. 3. Problem Representation Design Patterns
    1. Design Pattern 5: Reframing
      1. Problem
      2. Solution
      3. Why It Works
      4. Tradeoffs and Alternatives
    2. Design Pattern 6: Multilabel
      1. Problem
      2. Solution
      3. Trade Offs
    3. Design Pattern 7: Ensembles
      1. Problem
      2. Solution
      3. Why It Works
      4. Tradeoffs and Alternatives
    4. Design Pattern 8: Cascade
      1. Problem
      2. Solution
      3. Tradeoffs and Alternatives
    5. Design Pattern 9: Neutral Class
      1. Problem
      2. Solution
      3. Why It Works
      4. Tradeoffs and Alternatives
    6. Design Pattern 10: Rebalancing
      1. Problem
      2. Solutions
      3. Tradeoffs and Alternatives
    7. Summary
  4. 4. Patterns That Modify Model Training
    1. Typical Training Loop
      1. Stochastic Gradient Descent
      2. Keras Training Loop
      3. Training Design Patterns
    2. Design Pattern 11: Useful Overfitting
      1. Problem
      2. Solution
      3. Why It Works
      4. Tradeoffs and Alternatives
    3. Design Pattern 12: Checkpoints
      1. Problem
      2. Solution
      3. Why It Works
      4. Tradeoffs and Alternatives
    4. Design Pattern 13: Transfer Learning
      1. Problem
      2. Solution
      3. Why It Works
      4. Tradeoffs and Alternatives
    5. Design Pattern 14: Distribution Strategy
      1. Problem
      2. Solution
      3. Why It Works
      4. Tradeoffs and Alternatives
    6. Design Pattern 15: Hyperparameter Tuning
      1. Problem
      2. Solution
      3. Why It Works
      4. Tradeoffs and Alternatives
    7. Summary
  5. 5. Design Patterns for Resilient Serving
    1. Design pattern 16: Stateless Serving Function
      1. Problem
      2. Solution
      3. Why It Works
      4. Tradeoffs and Alternatives
    2. Design pattern 17: Batch Serving
      1. Problem
      2. Solution
      3. Why It Works
      4. Tradeoffs and Alternatives
    3. Design pattern 18: Continued Model Evaluation
      1. Problem
      2. Solution
      3. Why It Works
      4. Tradeoffs and Alternatives
    4. Design pattern 19: Two Phase Predictions
      1. Problem
      2. Solution
      3. Tradeoffs and Alternatives
    5. Design pattern 20: Keyed Predictions
      1. Problem
      2. Solution
      3. Tradeoffs and Alternatives
    6. Summary

Product Information

  • Title: Machine Learning Design Patterns
  • Author(s): Valliappa Lakshmanan, Sara Robinson, Michael Munn
  • Release date: December 2020
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781098115784