book

Data Lakehouse in Action

by Pradeep Menon

March 2022

Beginner to intermediate

206 pages

4h 21m

English

Packt Publishing

Read now

Unlock full access

ContributorsAbout the authorAbout the reviewer
Who this book is forWhat this book coversDownload the color imagesConventions usedGet in touchShare your thoughts
Discovering the enterprise data warehouse eraExploring the five factors of changeThe exponential growth of dataThe increase in computeThe decrease in storage costThe rise of artificial intelligenceThe advancement of cloud computing Investigating the data lake eraIntroducing the data lakehouse paradigmSummaryFurther reading
Developing a system context for a data lakehouseData providersData consumersDeveloping a logical data lakehouse architectureData ingestion layerData lake layerData processing layerData serving layerData analytics layerData governance layerData security layerDeveloping architecture principlesDisciplined at the core, flexible at the edgesDecouple compute and storageFocus on functionality rather than technologyCreate a modular architecturePerform active catalogingSummaryFurther reading
Ingesting and processing batch dataDifferences between the ETL and ELT patternsBatch data processing in a data lakehouseIngesting and processing streaming dataStreaming data sourcesExtraction-loadTransform-loadBringing it all togetherThe batch layerThe speed layerThe serving layerSummaryFurther reading
Storing data in the data lake layerData lake layerCommon data formatsStoring data in the data serving layerSQL-based serving NoSQL-based servingData-sharing technologySummaryFurther reading
Discussing the themes of analytics capabilitiesDescriptive analyticsAdvanced analyticsEnabling analytics capabilities in a data lakehouseThe analytics sandbox serviceThe business intelligence serviceThe AI serviceSummaryFurther reading
The 3-3-3 framework for data governance The three objectives of data governanceThe three pillars of data governanceThe three components of the data governance layerImplementing data governance policy managementImplementing the data catalogImplementing data qualitySummaryFurther reading

Realizing the data security components in a data lakehouseUsing IAM in a data lakehouseMethods of data encryption in a data lakehouseMethods of data masking in a data lakehouseMethods of implementing network security in a data lakehouseSummaryFurther reading
Why is cloud computing apt for implementing a data lakehouse?The rapid advancements in cloud computing facilitate data analyticsArchitectural flexibility is native to the cloudCloud computing enables tailored cost controlImplementing a data lakehouse on Microsoft AzureThe data ingestion layer on Microsoft AzureThe data processing layer on Microsoft AzureSummaryFurther reading
The need for a macro-architectural pattern for analyticsImplementing a data lakehouse in a macro-architectural patternThe hub-spoke patternThe data mesh patternChoosing between hub-spoke and data meshSummaryFurther reading
Other Books You May EnjoyPackt is searching for authors like youShare your thoughts

Content preview from Data Lakehouse in Action

Chapter 1: Introducing the Evolution of Data Analytics Patterns

Data analytics is an ever-changing field. A little history will help you appreciate the strides in this field and how data architectural patterns have evolved to fulfill the ever-changing need for analytics.

First, let's start with some definitions:

What is analytics? Analytics is defined as any action that converts data into insights.
What is data architecture? Data architecture is the structure that enables the storage, transformation, exploitation, and governance of data.

Analytics and the data architecture that enables analytics goes a long way. Let's now explore some of the patterns that have evolved over the last few decades.

This chapter explores the genesis of data growth ...