Automating Data Quality Monitoring

Chapter 6. Implementing Notifications While Avoiding Alert Fatigue

What good would a monitoring system be if it didn’t alert you when something went wrong? Yet alerts are both the essence and the Achilles’ heel of monitoring. Done right, notifications will reach the correct people via the appropriate channels, help them understand the severity of the issue, and even point them to the root cause for fast resolution. Done wrong, notifications will spam too many people, lack context, and be difficult to act on. They’ll be untrustworthy—and ultimately ignored.

Alert fatigue is one of the greatest reasons that data quality monitoring efforts fail, especially as enterprises scale (see the sidebar “Alert Fatigue”). If you have dozens of people involved in data quality and thousands of tables triggering alerts, that’s a lot of employee time and energy being wasted if those alerts are unhelpful, confusing, or noisy.

In this chapter, we’ll share a variety of techniques you can use to ensure your notifications are empowering your team to fix data quality issues. We’ll cover the steps in the issue resolution process and how notifications can be designed to help at each stage, even with root cause analysis. We’ll talk about how you find the right audience for any given notification and the right delivery mechanism. And we’ll explore many strategies for avoiding alert fatigue so that your system respects what may be our most valuable human resource: our attention.

How Notifications Facilitate ...