97 Things About Ethics Everyone in Data Science Should Know

Chapter 46. Toward Value-Based Machine Learning

Ron Bodkin

Office of the CTO, Google Cloud

Machine learning (ML) has become integral to many aspects of modern life, as digital experiences proliferate and we increasingly rely on automated algorithms for discovery, curation, and guiding our choices in areas as diverse as entertainment content (e.g., Medium and TikTok), communication (Slack and Gmail), navigation (Google Maps), and shopping (Amazon and Stitch Fix).

ML is often viewed as a value-neutral technology and as objective, unaligned with or dependent on values. But the reality is that ML is a tool—and like any tool, its use is based on values, and the consequences it creates impact our values.

I have been responsible for applying ML to real-world problems since 2007 and have repeatedly found that the use of ML leads to unintended consequences. Much like an evil genie, ML models will often grant exactly what you wished for (optimize what you specify) but not what you really intended. Ten years ago, when I was Vice President of Engineering at Quantcast, we would often be frustrated to see that ML models we created didn’t work properly. They would exploit subtle errors in our data or problem setup, and we had to work hard to understand what made them work so we could fix our data and fix our objectives (or loss functions) to achieve the results we intended.

More recently, there ...