Data Governance with Unity Catalog on Databricks

Chapter 8. Data Sharing and Collaboration

Homo sapiens rules the world because it is the only animal that can cooperate flexibly in large numbers.

Yuval Noah Harari (Sapiens: A Brief History of Humankind)

In his countless conversations with customers, Karthik began to detect a pattern: decision makers across organizations, especially large ones with a presence across different geographies, tended to look for a particular criterion in data platforms: how easy it was to securely share data and collaborate.

This pattern has two main aspects to it: one is about publishing assets targeting one or more consumers. The other is about sharing data and other assets for collaborating on business use cases, which can be as simple as sharing what you own with your teammates, or as complex as collaborating with other organizations across geographies on a use case that involves sensitive data. The data platforms that offer these features are favored by the decision makers. Most organizations are paying the price for not having considered these aspects in the past, which has led to silos resulting in a lack of collaboration and duplication of assets and efforts.

We are indeed happy that organizations are on the right path. However, the road to success with data sharing and collaboration is a hard one, especially in terms of security and governance. Before we discuss the associated challenges, let’s first clarify some of the terms that we use.

Similar to the term data governance, which we extended ...