97 Things About Ethics Everyone in Data Science Should Know

Chapter 60. The Before, Now, and After of Ethical Systems

Evan Stubbs

Partner and Associate Director, Boston Consulting Group

Closed-loop systems that use data to fuel artificial intelligence have the ability to change the world. Such systems are already saving lives, allocating capital, executing contracts, and making increasing numbers of decisions on behalf of their human overlords. As an architect of the future, wielding this power to make the world a better rather than a lesser place demands respect and consideration. Much like privacy or security, ethics in these systems can’t be treated as an aside. It needs to be treated as an integral part of design right from inception through to execution and beyond.

In a very practical sense, there are three points in every engineer’s journey at which poor choices can lead to perverse outcomes: the before, the now, and the after.

All of the following examples are real, albeit with the names withheld to protect the guilty.

During the before point, the firmament has yet to be built. Here, the biggest risk is unintentionally causing personal or social harm despite the best of intentions; algorithms are trained on data, and if that data reflects institutionalized disadvantage, the future will be forced to mirror the present.

Consider a firm that’s interested in using AI to identify and accelerate high performers within its staff. To train ...