book

Data Lakehouse in Action

by Pradeep Menon

March 2022

Beginner to intermediate

206 pages

4h 21m

English

Packt Publishing

Read now

Unlock full access

ContributorsAbout the authorAbout the reviewer
Who this book is forWhat this book coversDownload the color imagesConventions usedGet in touchShare your thoughts
Discovering the enterprise data warehouse eraExploring the five factors of changeThe exponential growth of dataThe increase in computeThe decrease in storage costThe rise of artificial intelligenceThe advancement of cloud computing Investigating the data lake eraIntroducing the data lakehouse paradigmSummaryFurther reading
Developing a system context for a data lakehouseData providersData consumersDeveloping a logical data lakehouse architectureData ingestion layerData lake layerData processing layerData serving layerData analytics layerData governance layerData security layerDeveloping architecture principlesDisciplined at the core, flexible at the edgesDecouple compute and storageFocus on functionality rather than technologyCreate a modular architecturePerform active catalogingSummaryFurther reading
Ingesting and processing batch dataDifferences between the ETL and ELT patternsBatch data processing in a data lakehouseIngesting and processing streaming dataStreaming data sourcesExtraction-loadTransform-loadBringing it all togetherThe batch layerThe speed layerThe serving layerSummaryFurther reading
Storing data in the data lake layerData lake layerCommon data formatsStoring data in the data serving layerSQL-based serving NoSQL-based servingData-sharing technologySummaryFurther reading
Discussing the themes of analytics capabilitiesDescriptive analyticsAdvanced analyticsEnabling analytics capabilities in a data lakehouseThe analytics sandbox serviceThe business intelligence serviceThe AI serviceSummaryFurther reading
The 3-3-3 framework for data governance The three objectives of data governanceThe three pillars of data governanceThe three components of the data governance layerImplementing data governance policy managementImplementing the data catalogImplementing data qualitySummaryFurther reading

Realizing the data security components in a data lakehouseUsing IAM in a data lakehouseMethods of data encryption in a data lakehouseMethods of data masking in a data lakehouseMethods of implementing network security in a data lakehouseSummaryFurther reading
Why is cloud computing apt for implementing a data lakehouse?The rapid advancements in cloud computing facilitate data analyticsArchitectural flexibility is native to the cloudCloud computing enables tailored cost controlImplementing a data lakehouse on Microsoft AzureThe data ingestion layer on Microsoft AzureThe data processing layer on Microsoft AzureSummaryFurther reading
The need for a macro-architectural pattern for analyticsImplementing a data lakehouse in a macro-architectural patternThe hub-spoke patternThe data mesh patternChoosing between hub-spoke and data meshSummaryFurther reading
Other Books You May EnjoyPackt is searching for authors like youShare your thoughts

Content preview from Data Lakehouse in Action

Chapter 2: The Data Lakehouse Architecture Overview

A well-thought-out architecture is the cornerstone of any robust information technology (IT) system, and a data lakehouse is no exception. The last chapter elucidated the need for a modern data analytics platform. The chapter also discussed the evolution of the data lakehouse. This chapter will focus on the critical elements of a data lakehouse.

The chapter will begin by describing the system context of a data lakehouse. Then, it will investigate the actors and systems that interact with a data lakehouse.

We will then discuss the logical architecture of a data lakehouse that consists of seven layers. The chapter will then deep-dive into various components of a data lakehouse architecture ...