Beautiful Data

Privacy Considerations

Smarter systems, like those capable of performing "data finds data," require very close attention to privacy and civil liberties protections. How these next-generation systems are built and deployed, and what policies (including accountability and oversight) govern their use deserves close attention and vigorous debate. Some of the core issues include: defining what data should be indexed for discoverability, how the data will be stitched together (e.g., what constitutes a relationship?), what constitutes relevance, what relevance is disclosed to whom, who can search the index, how the system will be monitored for unauthorized use, and how errors will be detected and corrected.

Fortunately, the directory-based model has a number of nice privacy-enhancing characteristics, including:

• Urges to share more data with more parties are replaced by transferring less information to fewer places (card catalogs).

• Who searches for what and what they found can be logged (for instance, using tamper-resistant logs) in a consistent manner, thus facilitating better accountability and oversight.^[12]

• Information sharing between parties is now reduced to just the records that they need to know and to share (sharing less by sharing only the information that must be shared).

• It is now possible to make the index anonymized, which means the risk of unintended disclosure of even the limited metadata in the index is drastically reduced.^[13]

^[12] Tamper-resistant logs are also often called ...