Responsible Machine Learning

Chapter 1. Introduction to Responsible Machine Learning

“Success in creating effective AI, could be the biggest event in the history of our civilization. Or the worst.”

Stephen Hawking

Machine learning (ML) systems can make and save money for organizations across industries, and they’re a critical aspect of many organization’s digital transformation plans. For these reasons (and others), ML investments were increasing rapidly before the COVID-19 crisis, and they’re expected to stay healthy as the situation unfolds. However, ML systems present risks for operators, consumers, and the general public. In many ways, this is similar to an older generation of transformational commercial technologies, like jetliners and nuclear reactors. Like these technologies, ML can fail on its own, or adversaries can attack it. Unlike some older transformational technologies, and despite growing evidence of ML’s capability to do serious harm, ML practitioners don’t seem to consider risk mitigation to be a primary directive of their work.¹

Common ML failure modes include unaccountable black-box mechanisms, social discrimination, security vulnerabilities, privacy harms, and the decay of system quality over time. Most ML attacks involve insider manipulation of training data and model mechanisms; manipulation of predictions or intellectual property extraction by external adversaries; or trojans hidden in third-party data, models, or other artifacts. When failures or attacks spiral out of control, they ...