Operationalizing the Data Lake

Chapter 1. The Data Lake: A Central Repository

In the introduction, we examined how companies are beginning to recognize the value of their data. Every business with information systems essentially uses a data lake, whether it is a staging environment for a data mart or a temporary storage layer for downstream processes like loading data warehouses or delivering to core systems.

These businesses are attempting to “activate” the data—that is, put it in the hands of the business users who need it—by taking advantage of the power of data lakes to operationalize their information in a way that can grow with the company.

These users almost immediately encounter several significant problems. Different users or teams might not be using the same versions of data. This happens when a dataset—for example, quarterly sales data—is split into or distributed to different data marts or other types of system silos in different departments. The data typically is cleaned, formatted, or changed in some way to fit these different types of users. Accounts payable and marketing departments may be looking at different versions of sales results. Each department may have their unconscious assumptions and biases that cause them to use the data in different ways. And sometimes the data itself is biased, which we look at more closely in the sidebar that follows.

The Four (Theoretical) Evils of Analytics

The four primary types of biases are information, confirmation, interpretation, and prediction. One or ...