97 Things Every Data Engineer Should Know

Chapter 58. Privacy Is Your Problem

Stephen Bailey, PhD

Data has been called “the new gold” for its ability to transform and automate business processes; it has also been called “the new uranium” for its ability to violate the human right to privacy on a massive scale. And just as nuclear engineers can effortlessly enumerate fundamental differences in gold and uranium, so too must data engineers learn to instinctively identify and separate dangerous data from the benign.

Take, for instance, the famous link attack that reidentified the medical records of several high-profile patients of Massachusetts General Hospital. In 1997, MGH released about 15,000 records in which names and patient IDs had been stripped from the database. Despite the precautions, Harvard researcher Latanya Sweeney was able to connect publicly available voter information to these anonymized medical records by joining them on three indirect identifiers: zip code, birthdate, and gender. This left Sweeney with only a handful of records to sift through to reidentify many individuals—most notably, the governor of Massachusetts patient records.

Twenty years later, every business is an MGH, and every person with internet access is a potential Latanya Sweeney. Yet we all want a world where data is handled responsibly, shared cautiously, and leveraged only for the right purposes. Our greatest limitation in realizing ...