To succeed with machine learning or deep learning, you must handle the logistics well. Simply put, you need an effective management system for overall data flow and the evaluation and deployment of multiple models as they move from prototype to production. Without that, your project will most likely fail. This report examines what you need for effective data and model management in real-world settings, including globally distributed cloud or on-premises systems.
Authors Ted Dunning and Ellen Friedman introduce the rendezvous architecture, an innovative design to help you handle machine-learning logistics. This approach not only paves the way to successful long-term management, it also frees up your time and effort to focus on the machine learning process itself and on how to take action on results.
This report provides a basic, non-technical view of what makes the approach work, as well as in-depth technical details. The report is ideal for data scientists, architects, developers, ops teams, and project managers, whether your team is planning to build a machine learning system, or currently has one underway.
You will learn:
- The issues in machine learning logistics you need to consider when designing and implementing your system
- How the rendezvous architecture leverages streaming data, provides hot hand-off of new models, and collects diagnostic data
- Practical tips for comparing live models, including the role of decoys, canaries and the t-digest
- Best practices for maintaining performance after deployment
Table of Contents
- 1. Why Model Management?
- 2. What Matters in Model Management
3. The Rendezvous Architecture for Machine Learning
- A Traditional Starting Point
- Why a Load Balancer Doesn’t Suffice
- A Better Alternative: Input Data as a Stream
- Message Contents
- The Decoy Model
- The Canary Model
- Adding Metrics
- Rule-Based Models
- Using Pre-Lined Containers
- 4. Managing Model Development
- 5. Machine Learning Model Evaluation
- 6. Models in Production
- 7. Meta Analytics
- 8. Lessons Learned
- A. Additional Resources
- Title: Machine Learning Logistics
- Release date: October 2017
- Publisher(s): O'Reilly Media, Inc.
- ISBN: 9781491997611