AI and Machine Learning for On-Device Development

Chapter 15. Ethics, Fairness, and Privacy for Mobile Apps

While recent advances in machine learning and AI have brought the concepts of ethics and fairness into the spotlight, it’s important to note that disparity and unfairness have always been topics of concern in computer systems. In my career, I have seen many examples where a system has been engineered for one scenario without considering the overall impact with regard to fairness and bias.

Consider this example: your company has a database of its customers and wants to target a marketing campaign to get more customers in a particular zip code where it has identified a growth opportunity. To do so, the company will send discount coupons to people in that zip code whom it has connected with, but who haven’t yet purchased anything. You could write SQL like this to identify these potential customers:

SELECT * from Customers WHERE ZIP=target_zip AND PURCHASES=0

This might seem to be perfectly sensible code. But consider the demographics of that zip code. What if the majority of people who live there are of a particular race or age? Instead of growing your customer base evenly, you could be overtargeting one segment of the population, or worse, discriminating against another by offering discounts to people of one race but not another. Over time, continually targeting like this could result in a customer base that is skewed against the demographics of society, ultimately painting your company into a corner of primarily serving ...