book

Analytics Engineering with SQL and dbt

by Rui Pedro Machado, Helder Russa

December 2023

Intermediate to advanced

324 pages

8h 23m

English

O'Reilly Media, Inc.

Book available

Read now

Unlock full access

Includes

Includes Quizzes

Why We Wrote This BookWho This Book Is ForHow This Book Is OrganizedConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
Databases and Their Impact on Analytics EngineeringCloud Computing and Its Impact on Analytics EngineeringThe Data Analytics LifecycleThe New Role of Analytics EngineerResponsibilities of an Analytics EngineerEnabling Analytics in a Data MeshData Productsdbt as a Data Mesh EnablerThe Heart of Analytics EngineeringThe Legacy ProcessesUsing SQL and Stored Procedures for ETL/ELTUsing ETL ToolsThe dbt RevolutionSummary
A Brief on Data ModelingThe Conceptual Phase of ModelingThe Logical Phase of ModelingThe Physical Phase of ModelingThe Data Normalization ProcessDimensional Data ModelingModeling with the Star SchemaModeling with the Snowflake SchemaModeling with Data VaultMonolith Data ModelingBuilding Modular Data ModelsEnabling Modular Data Models with dbtTesting Your Data ModelsGenerating Data DocumentationDebugging and Optimizing Data ModelsMedallion Architecture PatternSummary
The Resiliency of SQLDatabase FundamentalsTypes of DatabasesDatabase Management System“Speaking” with a DatabaseCreating and Managing Your Data Structures with DDLManipulating Data with DMLInserting Data with INSERTSelecting Data with SELECTUpdating Data with UPDATEDeleting Data with DELETEStoring Queries as ViewsCommon Table ExpressionsWindow FunctionsSQL for Distributed Data ProcessingData Manipulation with DuckDBData Manipulation with PolarsData Manipulation with FugueSQLBonus: Training Machine Learning Models with SQLSummary
dbt Design Philosophydbt Data Flowdbt CloudSetting Up dbt Cloud with BigQuery and GitHubUsing the dbt Cloud UIUsing the dbt Cloud IDEStructure of a dbt ProjectJaffle Shop DatabaseYAML FilesModelsSourcesTestsAnalysesSeedsDocumentationdbt Commands and Selection SyntaxJobs and DeploymentSummary
Model MaterializationsTables, Views, and Ephemeral ModelsIncremental ModelsMaterialized ViewsSnapshotsDynamic SQL with JinjaUsing SQL Macrosdbt PackagesInstalling PackagesExploring the dbt_utils PackageUsing Packages Inside Macros and Modelsdbt Semantic LayerSummary
Problem Definition: An Omnichannel Analytics CaseOperational Data ModelingConceptual ModelLogical ModelPhysical ModelHigh-Level Data ArchitectureAnalytical Data ModelingIdentify the Business ProcessesIdentify Facts and Dimensions in the Dimensional Data ModelIdentify the Attributes for DimensionsDefine the Granularity for Business FactsCreating Our Data Warehouse with dbtTests, Documentation, and Deployment with dbtData Analytics with SQLConclusion

Content preview from Analytics Engineering with SQL and dbt

Chapter 2. Data Modeling for Analytics

In today’s data-driven world, organizations rely more and more on data analytics to gain valuable insights and make informed decisions. Data modeling plays an imperative role in this process, providing a solid foundation for structuring and organizing data to support effective analysis. In addition, understanding the concepts of data modeling and normalization is essential to realizing the full potential of analytics and gaining actionable insights from complex datasets.

Data modeling is about defining the structure, relationships, and attributes of data entities within a system. An essential aspect of data modeling is the normalization of the data. Data normalization is a technique for eliminating data redundancy and improving data integrity. It involves breaking data into logical units and organizing them into separate tables, which reduces data duplication and improves overall database efficiency. Normalization ensures that data is stored in a structured and consistent manner, which is critical for accurate analysis and reliable results.

Regarding analytics, data modeling provides a solid foundation for creating analytical models. Analysts can design effective models that capture relevant information and support the desired analytics objectives by understanding the relationships among entities and data structures. In other words, a well-designed data model enables analysts to perform complex queries, join tables, and aggregate data to ...