Machine Learning Design Patterns

Machine Learning Design Patterns

by Valliappa Lakshmanan, Sara Robinson, Michael Munn
Released October 2020
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781098115784

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Buy on Amazon Buy on ebooks.com
Start your free trial

Book description

The design patterns in this book capture best practices and solutions to recurring problems in machine learning. The authors, three Google engineers, catalog proven methods to help data scientists tackle common problems throughout the ML process. These design patterns codify the experience of hundreds of experts into straightforward, approachable advice.

In this book, you will find detailed explanations of 30 patterns for data and problem representation, operationalization, repeatability, reproducibility, flexibility, explainability, and fairness. Each pattern includes a description of the problem, a variety of potential solutions, and recommendations for choosing the best technique for your situation.

You'll learn how to:

  • Identify and mitigate common challenges when training, evaluating, and deploying ML models
  • Represent data for different ML model types, including embeddings, feature crosses, and more
  • Choose the right model type for specific problems
  • Build a robust training loop that uses checkpoints, distribution strategy, and hyperparameter tuning
  • Deploy scalable ML systems that you can retrain and update to reflect new data
  • Interpret model predictions for stakeholders and ensure models are treating users fairly

Publisher resources

View/Submit Errata

Table of contents

  1. Preface
    1. Who Is This Book For?
    2. What’s Not in the Book
    3. Code Samples
    4. Conventions Used in This Book
    5. O’Reilly Online Learning
    6. How to Contact Us
    7. Acknowledgments
  2. 1. The Need for Machine Learning Design Patterns
    1. What Are Design Patterns?
    2. How to Use This Book
    3. Machine Learning Terminology
      1. Models and Frameworks
      2. Data and Feature Engineering
      3. The Machine Learning Process
      4. Data and Model Tooling
      5. Roles
    4. Common Challenges in Machine Learning
      1. Data Quality
      2. Reproducibility
      3. Data Drift
      4. Scale
      5. Multiple Objectives
    5. Summary
  3. 2. Data Representation Design Patterns
    1. Simple Data Representations
      1. Numerical Inputs
      2. Categorical Inputs
    2. Design Pattern 1: Hashed Feature
      1. Problem
      2. Solution
      3. Why It Works
      4. Trade-Offs and Alternatives
    3. Design Pattern 2: Embeddings
      1. Problem
      2. Solution
      3. Why It Works
      4. Trade-Offs and Alternatives
    4. Design Pattern 3: Feature Cross
      1. Problem
      2. Solution
      3. Why It Works
      4. Trade-Offs and Alternatives
    5. Design Pattern 4: Multimodal Input
      1. Problem
      2. Solution
      3. Trade-Offs and Alternatives
    6. Summary
  4. 3. Problem Representation Design Patterns
    1. Design Pattern 5: Reframing
      1. Problem
      2. Solution
      3. Why It Works
      4. Trade-Offs and Alternatives
    2. Design Pattern 6: Multilabel
      1. Problem
      2. Solution
      3. Trade-Offs and Alternatives
    3. Design Pattern 7: Ensembles
      1. Problem
      2. Solution
      3. Why It Works
      4. Trade-Offs and Alternatives
    4. Design Pattern 8: Cascade
      1. Problem
      2. Solution
      3. Trade-Offs and Alternatives
    5. Design Pattern 9: Neutral Class
      1. Problem
      2. Solution
      3. Why It Works
      4. Trade-Offs and Alternatives
    6. Design Pattern 10: Rebalancing
      1. Problem
      2. Solution
      3. Trade-Offs and Alternatives
    7. Summary
  5. 4. Model Training Patterns
    1. Typical Training Loop
      1. Stochastic Gradient Descent
      2. Keras Training Loop
      3. Training Design Patterns
    2. Design Pattern 11: Useful Overfitting
      1. Problem
      2. Solution
      3. Why It Works
      4. Trade-Offs and Alternatives
    3. Design Pattern 12: Checkpoints
      1. Problem
      2. Solution
      3. Why It Works
      4. Trade-Offs and Alternatives
    4. Design Pattern 13: Transfer Learning
      1. Problem
      2. Solution
      3. Why It Works
      4. Trade-Offs and Alternatives
    5. Design Pattern 14: Distribution Strategy
      1. Problem
      2. Solution
      3. Why It Works
      4. Trade-Offs and Alternatives
    6. Design Pattern 15: Hyperparameter Tuning
      1. Problem
      2. Solution
      3. Why It Works
      4. Trade-Offs and Alternatives
    7. Summary
  6. 5. Design Patterns for Resilient Serving
    1. Design Pattern 16: Stateless Serving Function
      1. Problem
      2. Solution
      3. Why It Works
      4. Trade-Offs and Alternatives
    2. Design Pattern 17: Batch Serving
      1. Problem
      2. Solution
      3. Why It Works
      4. Trade-Offs and Alternatives
    3. Design Pattern 18: Continued Model Evaluation
      1. Problem
      2. Solution
      3. Why It Works
      4. Trade-Offs and Alternatives
    4. Design Pattern 19: Two-Phase Predictions
      1. Problem
      2. Solution
      3. Trade-Offs and Alternatives
    5. Design Pattern 20: Keyed Predictions
      1. Problem
      2. Solution
      3. Trade-Offs and Alternatives
    6. Summary
  7. 6. Reproducibility Design Patterns
    1. Design Pattern 21: Transform
      1. Problem
      2. Solution
      3. Trade-Offs and Alternatives
    2. Design Pattern 22: Repeatable Splitting
      1. Problem
      2. Solution
      3. Trade-Offs and Alternatives
    3. Design Pattern 23: Bridged Schema
      1. Problem
      2. Solution
      3. Trade-Offs and Alternatives
    4. Design Pattern 24: Windowed Inference
      1. Problem
      2. Solution
      3. Trade-Offs and Alternatives
    5. Design Pattern 25: Workflow Pipeline
      1. Problem
      2. Solution
      3. Why It Works
      4. Trade-Offs and Alternatives
    6. Design Pattern 26: Feature Store
      1. Problem
      2. Solution
      3. Why It Works
      4. Trade-Offs and Alternatives
    7. Design Pattern 27: Model Versioning
      1. Problem
      2. Solution
      3. Trade-Offs and Alternatives
    8. Summary
  8. 7. Responsible AI
    1. Design Pattern 28: Heuristic Benchmark
      1. Problem
      2. Solution
      3. Trade-Offs and Alternatives
    2. Design Pattern 29: Explainable Predictions
      1. Problem
      2. Solution
      3. Trade-Offs and Alternatives
    3. Design Pattern 30: Fairness Lens
      1. Problem
      2. Solution
      3. Trade-Offs and Alternatives
    4. Summary
  9. 8. Connected Patterns
    1. Patterns Reference
    2. Pattern Interactions
    3. Patterns Within ML Projects
      1. ML Life Cycle
      2. AI Readiness
    4. Common Patterns by Use Case and Data Type
      1. Natural Language Understanding
      2. Computer Vision
      3. Predictive Analytics
      4. Recommendation Systems
      5. Fraud and Anomaly Detection
  10. Index

Product information

  • Title: Machine Learning Design Patterns
  • Author(s): Valliappa Lakshmanan, Sara Robinson, Michael Munn
  • Release date: October 2020
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781098115784