O'Reilly logo

Pervasive Intelligence Now by Anu Jain

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

CHAPTER 8 The Evolution of Data Lakes as a Fundamental Part of a New Ecosystem Architecture

Pervasive intelligence requires an intelligent, reliable information architecture. The foundation of an intelligent information architecture is the enterprise data warehouse (DW)—or at a minimum, function-oriented DWs that rely on the same governance structure and data definitions. Traditional DW implementations follow a set approach: define the requirements, identify (structured) data sources, define the schema, load and format the data, then distribute the data through presentation layers.

You can use historical transaction or event data in your DW to figure out what happened and why it might have happened. Based on available data, you can form hypotheses and test them against the data in the warehouse to either confirm or refute your suspicions. It can democratize BI (business intelligence) and put the power of analysis in the hands of less-than-tech-savvy users, presenting them with a single version of the “truth,” no matter who’s asking.

CHANGES IN DATA DICTATE CHANGES IN STORAGE TECHNOLOGY

Data warehouses work well as long as the data fits what you’ve defined, but no one needs to tell you that the nature of data has changed significantly over the past decade. The volume of unstructured and streaming data has far surpassed that of traditional, structured data. That flood has given rise to a new repository: the data lake.

Data lakes store unstructured data and format it when it’s ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required