book

Data Lakehouse in Action

by Pradeep Menon

March 2022

Beginner to intermediate

206 pages

4h 21m

English

Packt Publishing

Read now

Unlock full access

ContributorsAbout the authorAbout the reviewer
Who this book is forWhat this book coversDownload the color imagesConventions usedGet in touchShare your thoughts
Discovering the enterprise data warehouse eraExploring the five factors of changeThe exponential growth of dataThe increase in computeThe decrease in storage costThe rise of artificial intelligenceThe advancement of cloud computing Investigating the data lake eraIntroducing the data lakehouse paradigmSummaryFurther reading
Developing a system context for a data lakehouseData providersData consumersDeveloping a logical data lakehouse architectureData ingestion layerData lake layerData processing layerData serving layerData analytics layerData governance layerData security layerDeveloping architecture principlesDisciplined at the core, flexible at the edgesDecouple compute and storageFocus on functionality rather than technologyCreate a modular architecturePerform active catalogingSummaryFurther reading
Ingesting and processing batch dataDifferences between the ETL and ELT patternsBatch data processing in a data lakehouseIngesting and processing streaming dataStreaming data sourcesExtraction-loadTransform-loadBringing it all togetherThe batch layerThe speed layerThe serving layerSummaryFurther reading
Storing data in the data lake layerData lake layerCommon data formatsStoring data in the data serving layerSQL-based serving NoSQL-based servingData-sharing technologySummaryFurther reading
Discussing the themes of analytics capabilitiesDescriptive analyticsAdvanced analyticsEnabling analytics capabilities in a data lakehouseThe analytics sandbox serviceThe business intelligence serviceThe AI serviceSummaryFurther reading
The 3-3-3 framework for data governance The three objectives of data governanceThe three pillars of data governanceThe three components of the data governance layerImplementing data governance policy managementImplementing the data catalogImplementing data qualitySummaryFurther reading

Realizing the data security components in a data lakehouseUsing IAM in a data lakehouseMethods of data encryption in a data lakehouseMethods of data masking in a data lakehouseMethods of implementing network security in a data lakehouseSummaryFurther reading
Why is cloud computing apt for implementing a data lakehouse?The rapid advancements in cloud computing facilitate data analyticsArchitectural flexibility is native to the cloudCloud computing enables tailored cost controlImplementing a data lakehouse on Microsoft AzureThe data ingestion layer on Microsoft AzureThe data processing layer on Microsoft AzureSummaryFurther reading
The need for a macro-architectural pattern for analyticsImplementing a data lakehouse in a macro-architectural patternThe hub-spoke patternThe data mesh patternChoosing between hub-spoke and data meshSummaryFurther reading
Other Books You May EnjoyPackt is searching for authors like youShare your thoughts

Content preview from Data Lakehouse in Action

Chapter 6: Applying Data Governance in the Data Lakehouse

The journey so far has covered five layers of Data Lakehouse. This chapter will focus on the final two layers, that is, data governance and data security layers. Data governance and security is an essential aspect of the modern data analytics platform. We will start this chapter by discussing the need for a data governance framework and how the 3-3-3 framework outlines the three components of data governance. The next section of the chapter will discuss implementing the data governance components in the Data Lakehouse. The sections will first cover implementing the three data governance components. Then, the sections will include key constructs for each of the data governance components ...