97 Things About Ethics Everyone in Data Science Should Know

Chapter 48. The Ethics of Communicating Machine Learning Predictions

Rado Kotorov

CEO, Trendalyze Inc.

People today are fascinated by the amazing computing power that we have. Computers can find information faster than humans, extract insights from data more precisely than many people, answer questions quicker than experts, play chess better than masters, and much more. People have built so much respect for, and trust in, machines that they often communicate machine-generated insights as facts.

In his article “The Median Isn’t the Message”, originally published in Discover magazine in 1985, the renowned evolutionary anthropologist Stephen Jay Gould first alerted us to the dangers and moral consequences of presenting statistical and machine learning predictions to ordinary people who do not have mathematical or scientific backgrounds. In the article, he describes his personal experience of being diagnosed with a deadly cancer and the doctor’s refusal to tell him his life expectancy. He did the research himself at the medical library at Harvard and learned that the median life expectancy was merely eight months: “So that’s why they didn’t give me anything to read,” he thought. “Then my mind started to work again, thank goodness.”

Gould goes on to explain why using the median, the average, or any other statistically derived prediction for communicating life expectancy of incurable ...