Effective Machine Learning Teams

Chapter 1. Challenges and Better Paths in Delivering ML Solutions

The most dangerous kind of waste is the waste we do not recognize.

Shigeo Shingo, leading expert on the Toyota Production System

Not everything that is faced can be changed, but nothing can be changed until it is faced.

James Baldwin, writer and playwright

Many individuals and organizations start their machine learning (ML) journey with high hopes, but the lived experiences of many ML practitioners tell us that the journey of delivering ML solutions is riddled with traps, detours, and sometimes even insurmountable barriers. When we peel back the hype and the glamorous claims of data science being the sexiest job of the 21st century, we often see ML practitioners bogged down by burdensome manual work; firefighting in production; team silos; and unwieldy, brittle, and complex solutions.

This hinders, or even prevents, teams from delivering value to customers and also frustrates an organization’s investments and ambitions in AI. As hype cycles go, many travel past the peak of inflated expectations and crash-land into the trough of disillusionment. We might see some high-performing ML teams move on to the plateau of productivity and wonder if we’ll ever get there.

Regardless of your background—be it academia, data science, ML engineering, product management, software engineering, or something else—if you are building products or systems that involve ML, you will inevitably face the challenges that we describe in this ...