Chapter 3. Data Warehouses Versus Data Lakes: A Primer
Chapters 1 and 2 introduced the idea of a data-driven organization and defined the concept of DataOps within the context of big data initiatives. Now, it’s time to take a step back and explore some other basic but important concepts. One of our most important tasks at this point is to clearly delineate the differences between data warehouses and data lakes.
When I give talks about self-service data, questions inevitably come up. What distinguishes a data lake from a data warehouse? Do I need to choose between them or do I need both? What are current best practices for setting up the relationship between a data warehouse and a data lake? This chapter answers these questions and more, and delves into a detailed explanation of why augmenting your existing data warehouse with a data lake is the best path to take given the current state of maturity of the various technologies.
Data Warehouse: A Definition
At its most basic, a data warehouse is a central repository for all the data that is collected in an organization’s business systems. Data is extracted, transformed, and loaded (known as ETL) into a data warehouse, which supports applications for reporting, analytics, and data mining on this extracted and curated dataset (Figure 3-1). The previous generation of data infrastructure centered on data warehouses and was based on technologies such as Teradata, Oracle, Neteeza, Greenplum, and Vertica, among others.
Figure 3-1.
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month, and much more.
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.
Julian F.
Head of Cybersecurity
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.
Addison B.
Field Engineer
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.
Amir M.
Data Platform Tech Lead
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.
