97 Things About Ethics Everyone in Data Science Should Know

Chapter 63. A Framework for Managing Ethics in Data Science: Model Risk Management

Doug Hague

Executive Director, School of Data Science at UNC Charlotte

As data scientists work to understand the ethics and implications of their models, a management framework is needed. Fortunately, the model risk management (MRM) framework emerging from the financial services industry may be expanded to include ethics. Models from various industries, including résumé screeners, recidivism models, and health care payment models, may be biased against various users or protected groups and have resulted in poor publicity for any corporation found to be using them. As data scientists develop methods to manage bias, MRM may be useful for documenting and ensuring best practices are followed. My focus here is on applying MRM processes to mathematical biases of a model; however, the MRM framework is also applicable when broadening to fairness and the overall ethical implications of data science.

In simple terms, MRM is a process that reviews and monitors model development and operations. It consists of examining data quality, mathematical soundness, quality of predictions, appropriate use, and ongoing monitoring, all through independent review and validation. In each of these areas, bias may creep into a model’s prediction.

Data

If data is biased at the start (as most data is), MRM has checks and balances ...