97 Things About Ethics Everyone in Data Science Should Know

Chapter 40. Pay Off Your Fairness Debt, the Shadow Twin of Technical Debt

Arnobio Morelix

Chief Innovation Officer, Startup Genome Data Scientist-in-Residence, Inc. Magazine

Technical debt is a familiar concept. It is used to describe hacky code created on the fly that does its primary job in the short term but is unwieldy and inefficient to maintain and scale in the long term. It is time we also become familiar with its shadow twin: fairness debt.

Just like its technical counterpart, we incur fairness debt when we build systems that work for our current situation and user base today but that have unintended consequences lurking underneath the surface as we continue to deploy the solutions tomorrow.

One way to incur fairness debt is by optimizing our systems and algorithms for a particular performance metric without constraints. Data scientists and technologists make these types of optimization choices deliberately and often, even if naively.

But optimization often carries a fairness debt when taken to its natural progression. A Google Ventures post, for example, suggests optimizing for the amount of time users spend watching videos on your app. While at first this may seem a perfectly rational way to focus engineering efforts, it can get out of control when usage becomes excessive, to the detriment of the user. As a friend managing AI products at Amazon said, “It is OK when a ...