Practical Fairness

Chapter 2. Understanding Fairness and the Data Science Pipeline

Now that you’ve read a little background on the history and social context of technology and fairness, let’s dig into the engineering aspects of defining fairness. In this chapter, I discuss how to set engineering goals and use fairness metrics. Of course, using metrics can be a dangerous exercise; after all, it’s a well-known problem that one gets what one measures. So let’s explore these metrics but also remember that no single metric can fully encompass or guarantee fairness.

What do we want our work to accomplish from a fairness perspective? What are our targets with respect to equity, privacy, and security? Should these targets be numerical quotas, or should we leave them deliberately underspecified so that we’re always striving to do better? To begin answering these questions, let’s start with some brief definitions of what we’re after, building on the observations in the previous chapter:

Equality

One version of a perfect world is one in which everyone has about the same and also deserves about the same. Houses would all be about the same size, and everyone would have the same amount of leisure. Everyone would have access to equally good health care and equally good education. Of course, this also supposes that individuals would be equally careful of their health, make equally good use of their educational opportunities, and take equally good care of their homes, which in the real world is not very likely. ...