book

Driving Data Quality with Data Contracts

Name: Driving Data Quality with Data Contracts
Author: Andrew Jones
ISBN: 9781837635009

by Andrew Jones

June 2023

Intermediate to advanced

206 pages

5h 33m

English

Packt Publishing

Read now

Unlock full access

Driving Data Quality with Data Contracts
ForewordContributorsAbout the authorAbout the reviewers
Preface
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesConventions usedGet in touchShare Your ThoughtsDownload a free PDF copy of this book
Part 1: Why Data Contracts?
Chapter 1: A Brief History of Data Platforms
The enterprise data warehouseThe big data platformThe modern data stackThe state of today’s data platformsThe lack of expectationsThe lack of reliabilityThe lack of autonomyThe ever-increasing use of data in business-critical applicationsSummaryFurther reading
Chapter 2: Introducing Data Contracts
What is a data contract?An agreed interface between the generators of data, and its consumersSetting expectations around that dataDefining how the data should be governedFacilitating the explicit generation of quality dataThe four principles of data contractsWhen to use data contractsData contracts and the data meshDomain ownershipData as a productSelf-serve data platformFederated computational governanceData contracts enable a data meshSummaryFurther reading
Part 2: Driving Data Culture Change with Data Contracts
Chapter 3: How to Get Adoption in Your Organization
Using data contracts to change an organizationArticulating the value of your dataBuilding data productsWhat is a data product?Adopting a data product mindsetDesigning a data productWalking through an example of a data productSummaryFurther reading
Chapter 4: Bringing Data Consumers and Generators Closer Together
Who is a consumer, and who is a generator?Data consumersData generatorsAssigning responsibility and accountabilityFeeding data back to the product teamsManaging the evolution of dataSummaryFurther reading
Chapter 5: Embedding Data Governance
Why we need data governanceThe requirements of data governanceHow data governance programs are typically appliedPromoting data governance through data contractsAssigning responsibility for data governanceResponsibilities of the data generatorsIntroducing the data architecture councilWorking together to implement federated data governanceSummaryFurther reading
Part 3: Designing and Implementing a Data Architecture Based on Data Contracts

Chapter 6: What Makes Up a Data Contract
The schema of a data contractDefining a schemaUsing a schema registry as the source of truthEvolving your data over timeEvolving your schemasMigrating your consumersDefining the governance and controlsSummaryFurther reading
Chapter 7: A Contract-Driven Data Architecture
A step-change in building data platformsBuilding generic data toolingIntroducing a data infrastructure teamA case study from GoCardless in promoting autonomyPromoting autonomy through decentralizationIntroducing the principles of a contract-driven data architectureAutomationGuidelines and guardrailsConsistencyProviding self-served data infrastructureSummaryFurther reading
Chapter 8: A Sample Implementation
Technical requirementsCreating a data contractProviding the interfaces to the dataIntroducing IaCCreating the interfaces from the data contractCreating libraries for data generatorsPopulating a central schema registryRegistering a schema with the Confluent schema registryManaging schema evolutionImplementing contract-driven toolingSummaryFurther reading
Chapter 9: Implementing Data Contracts in Your Organization
Getting started with data contractsThe ability to define a data contractThe ability to provision an interface for the data for consumers to queryThe ability of generators to write data to the interfaceMigrating to data contractsDiscovering data contractsWhat is a data catalog?Why are data catalogs important for discovering data contracts?What is data lineage?Why is data lineage important for data contracts?Building a mature data contracts-backed data cultureSummaryFurther reading
Chapter 10: Data Contracts in Practice
Designing a data contractIdentifying the purposeConsidering the trade-offsDefining the data contractDeploying the data contractMonitoring and enforcing data contractsThe data contract’s definitionThe quality of the dataThe performance and dependability of the dataData contract publishing patternsWriting directly to the interfaceMaterialized views on CDCThe transactional outbox patternThe listen-to-yourself patternSummaryFurther reading
Index
Why subscribe?
Other Books You May EnjoyPackt is searching for authors like youShare Your ThoughtsDownload a free PDF copy of this book

Content preview from Driving Data Quality with Data Contracts

Part 3: Designing and Implementing a Data Architecture Based on Data Contracts

In this part, we’ll look at exactly how to design and implement data contracts, including a sample implementation, and provide practical advice to implement data contracts in your organization.

This part comprises the following chapters:

Chapter 6, What Makes Up a Data Contract
Chapter 7, A Contract-Driven Data Architecture
Chapter 8, A Sample Implementation
Chapter 9, Implementing Data Contracts in Your Organization
Chapter 10, Data Contracts in Practice

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781837635009

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Driving Data Quality with Data Contracts

by Andrew Jones

Part 3: Designing and Implementing a Data Architecture Based on Data Contracts

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.