book

Data Lakehouse in Action

by Pradeep Menon

March 2022

Beginner to intermediate

206 pages

4h 21m

English

Packt Publishing

Read now

Unlock full access

ContributorsAbout the authorAbout the reviewer
Who this book is forWhat this book coversDownload the color imagesConventions usedGet in touchShare your thoughts
Discovering the enterprise data warehouse eraExploring the five factors of changeThe exponential growth of dataThe increase in computeThe decrease in storage costThe rise of artificial intelligenceThe advancement of cloud computing Investigating the data lake eraIntroducing the data lakehouse paradigmSummaryFurther reading
Developing a system context for a data lakehouseData providersData consumersDeveloping a logical data lakehouse architectureData ingestion layerData lake layerData processing layerData serving layerData analytics layerData governance layerData security layerDeveloping architecture principlesDisciplined at the core, flexible at the edgesDecouple compute and storageFocus on functionality rather than technologyCreate a modular architecturePerform active catalogingSummaryFurther reading
Ingesting and processing batch dataDifferences between the ETL and ELT patternsBatch data processing in a data lakehouseIngesting and processing streaming dataStreaming data sourcesExtraction-loadTransform-loadBringing it all togetherThe batch layerThe speed layerThe serving layerSummaryFurther reading
Storing data in the data lake layerData lake layerCommon data formatsStoring data in the data serving layerSQL-based serving NoSQL-based servingData-sharing technologySummaryFurther reading
Discussing the themes of analytics capabilitiesDescriptive analyticsAdvanced analyticsEnabling analytics capabilities in a data lakehouseThe analytics sandbox serviceThe business intelligence serviceThe AI serviceSummaryFurther reading
The 3-3-3 framework for data governance The three objectives of data governanceThe three pillars of data governanceThe three components of the data governance layerImplementing data governance policy managementImplementing the data catalogImplementing data qualitySummaryFurther reading

Realizing the data security components in a data lakehouseUsing IAM in a data lakehouseMethods of data encryption in a data lakehouseMethods of data masking in a data lakehouseMethods of implementing network security in a data lakehouseSummaryFurther reading
Why is cloud computing apt for implementing a data lakehouse?The rapid advancements in cloud computing facilitate data analyticsArchitectural flexibility is native to the cloudCloud computing enables tailored cost controlImplementing a data lakehouse on Microsoft AzureThe data ingestion layer on Microsoft AzureThe data processing layer on Microsoft AzureSummaryFurther reading
The need for a macro-architectural pattern for analyticsImplementing a data lakehouse in a macro-architectural patternThe hub-spoke patternThe data mesh patternChoosing between hub-spoke and data meshSummaryFurther reading
Other Books You May EnjoyPackt is searching for authors like youShare your thoughts

Content preview from Data Lakehouse in Action

Chapter 5: Deriving Insights from a Data Lakehouse

A lot of ground has been covered so far. The previous chapters covered the methods of ingesting, processing, storing, and serving data in a data lakehouse. Transforming the underlying data into insights is the core aim of any data analytics platform, so this chapter will focus on how to do this. We will also explore the different kinds of data analytics that can be employed in this process.

First, we'll discuss some of the business requirements relating to data analytics. Then, we'll explore different kinds of data analytics and how different stakeholders can use them. After that, we will dive into how these capabilities are enabled using the three components of the data analytics layer: the ...