Book description
Quickly detect, troubleshoot, and prevent a wide range of data issues through data observability, a set of best practices that enables data teams to gain greater visibility of data and its usage. If you're a data engineer, data architect, or machine learning engineer who depends on the quality of your data, this book shows you how to focus on the practical aspects of introducing data observability in your everyday work.
Author Andy Petrella helps you build the right habits to identify and solve data issues, such as data drifts and poor quality, so you can stop their propagation in data applications, pipelines, and analytics. You'll learn ways to introduce data observability, including setting up a framework for generating and collecting all the information you need.
- Learn the core principles and benefits of data observability
- Use data observability to detect, troubleshoot, and prevent data issues
- Follow the book's recipes to implement observability in your data projects
- Use data observability to create a trustworthy communication framework with data consumers
- Learn how to educate your peers about the benefits of data observability
Publisher resources
Table of contents
- Preface
- I. Introducing Data Observability
- 1. Introducing Data Observability
- 2. Components of Data Observability
- 3. Roles of Data Observability in a Data Organization
- II. Implementing Data Observability
-
4. Generate Data Observations
- At the Source
- Generating Data Observations at the Source
-
Low-Level API in Python
- Description of the Data Pipeline
- Definition of the Status of the Data Pipeline
- Data Observations for the Data Pipeline
- Generate Contextual Data Observations
- Generate Data-Related Observations
- Generate Lineage-Related Data Observations
- Wrap-Up: The Data-Observable Data Pipeline
- Using Data Observations to Address Failures of the Data Pipeline
- Conclusion
- 5. Automate the Generation of Data Observations
- 6. Implementing Expectations
- III. Data Observability in Action
- 7. Integrating Data Observability in Your Data Stack
- 8. Making Opaque Systems Translucent
- Afterword: Future Observations
- Index
- About the Author
Product information
- Title: Fundamentals of Data Observability
- Author(s):
- Release date: August 2023
- Publisher(s): O'Reilly Media, Inc.
- ISBN: 9781098133290
You might also like
book
Fundamentals of Data Engineering
Data engineering has grown rapidly in the past decade, leaving many software engineers, data scientists, and …
audiobook
Fundamentals of Data Engineering
Data engineering has grown rapidly in the past decade, leaving many software engineers, data scientists, and …
book
Practical Statistics for Data Scientists, 2nd Edition
Statistical methods are a key part of data science, yet few data scientists have formal statistical …
book
Data Science from Scratch, 2nd Edition
To really learn data science, you should not only master the tools—data science libraries, frameworks, modules, …