Practicing Trustworthy Machine Learning

Chapter 1. Privacy

If you’ve been paying any attention to the media, then you’re at least somewhat aware of the damage that can follow when a company’s customer data or proprietary algorithms are leaked. Given that the field of machine learning (ML) requires enormous amounts of data almost by definition, the risk is especially glaring.

Attack Vectors for Machine Learning Pipelines

Shortly after computers were invented, methods for attacking them were invented. To illustrate this, the MITRE corporation has created a taxonomy of tactics and techniques used by hackers to attack systems.

The emergence of machine learning created a bunch of additional ways in which computer systems could be attacked. In fact, there’s a machine learning–specific version of MITRE ATT&CK: MITRE ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems). Just as attackers and adversaries have sought to steal data from and control computer systems in general, machine learning pipelines are faced with the same risks.

This chapter goes into a series of techniques and technologies that can mitigate the risk of privacy leaks. While these techniques represent the intersection of practical best practices and state-of-the-art research, no tool is perfect. Some of these technologies can backfire if not properly implemented or if you focus on only one definition of privacy.

Improperly Implemented Privacy Features in ML: Case Studies

Before we dive into mathematical privacy definitions, let’s first ...