Skip to Main Content
Fundamentals of Data Observability
book

Fundamentals of Data Observability

by Andy Petrella
August 2023
Beginner to intermediate content levelBeginner to intermediate
264 pages
7h 15m
English
O'Reilly Media, Inc.
Book available
Content preview from Fundamentals of Data Observability

Chapter 6. Implementing Expectations

Like Chapter 2, we end this section on implementation with expectations. As data practitioners, expectations are key to the success of our data initiatives, especially their operational maintenance and keeping our stakeholders’ trust. Introducing them in our applications is an important step in their development and maintenance, just as important as tests and documentation.

This chapter highlights moments of the application lifecycle where we must introduce expectations and the associated practices to discover and implement them. Expectations also represent pieces of knowledge or interpretations of the constraints based on understanding the context. They have their maintenance process just as tests and documentation do.

To conclude the chapter, we present two examples of overarching practices that combine the benefits of generating observations at the source and introducing expectations. But first, let’s see how to define expectations and add them to our data applications.

Introducing Expectations

As mentioned in Chapter 2, expectations represent data constraints or usage that we must respect. If we don’t follow them, we must be aware and take action. This section explains how to turn the expectations into rules—how to define and implement them—and the actions we need to take.

Before jumping in, let’s see what a rule looks like. You can implement a rule as a function that takes observations as inputs and computes a Boolean condition in its ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Architecting Data and Machine Learning Platforms

Architecting Data and Machine Learning Platforms

Marco Tranquillin, Valliappa Lakshmanan, Firat Tekiner
Fundamentals of Data Engineering

Fundamentals of Data Engineering

Joe Reis, Matt Housley
Fundamentals of Data Engineering

Fundamentals of Data Engineering

Joe Reis, Matt Housley

Publisher Resources

ISBN: 9781098133283Errata PageSupplemental Content