Operationalizing the Data Lake

Chapter 2. The Importance of Building a Self-Service Culture

Before we can talk about how to build a data lake, we need to discuss the culture of the company using that data lake and, more specifically, the mental shift required for organizations to fully embrace the value of a data lake. In more traditional organizations, the DataOps team stands between the data and the business users. Traditionally, when a user needs data, they approach a data analyst or data scientist and make a request. The data team then responds. It’s common for the data team to build dashboards and have a set of prebuilt reports that it refreshes or sends out periodically, but so-called ad hoc requests are usually handled on a case-by-case basis.

This gatekeeper approach inevitably causes bottlenecks, as shown in Figure 2-1. Users who need data for a key presentation or to make a strategic decision are forced to wait for their turns in the queue. Often, they give up, and make the decision without having the data to back it up. And it becomes difficult, if not impossible, for an organization to extract the full value from its data using this paradigm. Because of this, a self-service culture is essential if your company is going to get the most value from your data and, eventually, become a true data-driven organization.

Figure 2-1. The importance of building a self-service culture

The End Goal: Becoming ...