97 Things About Ethics Everyone in Data Science Should Know

Chapter 83. Using Social Feedback Loops to Navigate Ethical Questions

Nick Hamlin

Data Scientist, GlobalGiving

Technological change is social change. As data-centric technology proliferates, product questions must be asked alongside social impact questions if companies hope to succeed in either area. These blurring lines also mean that data scientists must emphasize the ethical implications of their expanding impact. While oaths, checklists, and communities of practice for ethical data science are critical,¹ these constructs omit a key component: the social feedback loops that allow the voices of affected communities to inform product decisions.

“Mechanistic” feedback loops, like the results of a reinforcement learning algorithm informing future training iterations, are common in data science. They’re technical constructs that amplify a dataset’s signals in service of better predictions. In contrast, we’re focused here on “social” feedback loops—the processes that emphasize voices in a community of users whose ideas, concerns, and input are key to effective navigation of ethical challenges.²

But social feedback loops are hard! Their nuance makes them time consuming to analyze, and they often include contradictory ideas. When users’ feedback clashes with the company’s bottom-line goals, financial incentives frequently win out. Organizations without a culture of listening and openness ...