Practical Fairness

Chapter 4. Fairness Pre-Processing

As discussed in the previous chapter, fairness can affect three stages of the data modeling pipeline. This chapter focuses on the earliest stage, adjusting the way that data is translated into inputs for a machine learning training process, also called pre-processing the data.

The advantages of pre-processing a data set are numerous. For starters, many regard this as the most flexible fairness intervention, because if done well, it can prevent downstream misuse or carelessness leading to discrimination. If the discrimination is removed from the data, there is less of a concern that naive or careless downstream users could go wrong. Additionally, some methods for pre-processing a data set are more intuitive and inspectable than are methods that act during model training (i.e., in-processing).

Because pre-processing is the earliest opportunity for intervening in the data modeling process,¹ pre-processing offers the most opportunities for downstream metrics. When pre-processing is the fairness intervention used in the data modeling pipeline,² fairness metrics can be applied at different stages along the pipeline. For example, we can separately measure both how the pre-processing reduces discrimination in the data and how the pre-processing affects potentially discriminatory outputs of the model trained on the data set.

Because fairness in machine learning remains a relatively young field without a clearly established canon, and because fairness ...