Machine Learning Logistics

Machine Learning Logistics

by Ted Dunning, Ellen Friedman
Released October 2017
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781491997611

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

To succeed with machine learning or deep learning, you must handle the logistics well. Simply put, you need an effective management system for overall data flow and the evaluation and deployment of multiple models as they move from prototype to production. Without that, your project will most likely fail. This report examines what you need for effective data and model management in real-world settings, including globally distributed cloud or on-premises systems.

Authors Ted Dunning and Ellen Friedman introduce the rendezvous architecture, an innovative design to help you handle machine-learning logistics. This approach not only paves the way to successful long-term management, it also frees up your time and effort to focus on the machine learning process itself and on how to take action on results.

This report provides a basic, non-technical view of what makes the approach work, as well as in-depth technical details. The report is ideal for data scientists, architects, developers, ops teams, and project managers, whether your team is planning to build a machine learning system, or currently has one underway.

You will learn:

  • The issues in machine learning logistics you need to consider when designing and implementing your system
  • How the rendezvous architecture leverages streaming data, provides hot hand-off of new models, and collects diagnostic data
  • Practical tips for comparing live models, including the role of decoys, canaries and the t-digest
  • Best practices for maintaining performance after deployment

Table of contents

  1. Preface
    1. How This Book is Organized
    2. Acknowledgments
  2. 1. Why Model Management?
    1. The Best Tool for Machine Learning
      1. Tools for Deep Learning
    2. Fundamental Needs Cut Across Different Projects
    3. Tensors in the Henhouse
      1. Defining the Problem and the Project Goal
    4. Real-World Considerations
      1. Myth of the Unitary Model
    5. What Should You Ask about Model Management?
  3. 2. What Matters in Model Management
    1. Ingredients of the Rendezvous Approach
    2. DataOps Provides Flexibility and Focus
    3. Stream-Based Microservices
    4. Streams Offer More
    5. Building a Global Data Fabric
    6. Making Life Predictable: Containers
    7. Canaries and Decoys
    8. Types of Machine Learning Applications
      1. Decisioning
      2. Search-Like
      3. Interactive
    9. Conclusion
  4. 3. The Rendezvous Architecture for Machine Learning
    1. A Traditional Starting Point
    2. Why a Load Balancer Doesn’t Suffice
    3. A Better Alternative: Input Data as a Stream
      1. Rendezvous Style
    4. Message Contents
      1. Request Specific Fields
      2. Output Specific Fields
      3. Data Format
      4. Stateful Models
    5. The Decoy Model
    6. The Canary Model
    7. Adding Metrics
      1. Anomaly Detection
    8. Rule-Based Models
    9. Using Pre-Lined Containers
  5. 4. Managing Model Development
    1. Investing in Improvements
      1. Build One to Throw Away
      2. Annotate and Document
    2. Gift Wrap Your Models
    3. Other Considerations
  6. 5. Machine Learning Model Evaluation
    1. Why Compare Instead of Evaluate Offline?
    2. The Importance of Quantiles
    3. Quantile Sketching with t-Digest
    4. The Rubber Hits the Road
  7. 6. Models in Production
    1. Life with a Rendezvous System
      1. Model Life Cycle
      2. Upgrading the Rendezvous System
    2. Beware of Hidden Dependencies
      1. A Simplified Example of Data Coupling
    3. Monitoring
  8. 7. Meta Analytics
    1. Basic Tools
      1. Event Rate Change Detection
      2. t-Digest for One-Dimensional Score Binning
      3. K-Means Binning
      4. Aggregated Metrics
      5. Latency Traces
    2. Data Monitoring: Distribution of the Inputs
      1. Methods Specific for Operational Monitoring
      2. Combining Alerts
  9. 8. Lessons Learned
    1. New Frontier
    2. Where We Go from Here
  10. A. Additional Resources
    1. Selected O’Reilly Publications by Ted Dunning and Ellen Friedman
    2. O’Reilly Publication by Ellen Friedman and Kostas Tzoumas

Product information

  • Title: Machine Learning Logistics
  • Author(s): Ted Dunning, Ellen Friedman
  • Release date: October 2017
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781491997611