97 Things About Ethics Everyone in Data Science Should Know

Chapter 24. Is It Wrong to Be Right?

Marty Ellingsworth

Senior Analyst, Celent

Better data, better models, and better decision engines get better results. Whether you hand train a model, use automated machine learning systems, or employ any of the newest “x”-learning nets with pretraining, does it really matter why your model performs so well as long as it has great performance? Should not the most accurate model be used everywhere and anywhere in all situations? That’s progress, right? Should your model be allowed to learn on its own to further this progress and self-calibrate for better personalization? Practically, if there is no harm, then there is no foul, right?

Academically, many argue it’s all about the lift, correct for the data. Professionally, the standards of care are still emerging. There are dozens of fields of practice in which better models and better data appear magically, regularly, and at scale, for better results—no questions asked—resulting in delighted users, inventors, and investors. Privacy rights advocates sound warnings.

Legally, we are in a technological sprint that is far ahead of existing laws and compliance regulations. Many of today’s controls emerged back in the 1970s and 1980s around fairness in employment, lending, and housing, and in the 1990s and early 2000s around fraud, theft, collusion, money laundering, and terrorism concerns. Known legal ...