Anomaly Detection for Monitoring

Chapter 1. Introduction

Wouldn’t it be amazing to have a system that warned you about new behaviors and data patterns in time to fix problems before they happened, or seize opportunities the moment they arise? Wouldn’t it be incredible if this system was completely foolproof, warning you about every important change, but never ringing the alarm bell when it shouldn’t? That system is the holy grail of anomaly detection. It doesn’t exist, and probably never will. However, we shouldn’t let imperfection make us lose sight of the fact that useful anomaly detection is possible, and benefits those who apply it appropriately.

Anomaly detection is a set of techniques and systems to find unusual behaviors and/or states in systems and their observable signals. We hope that people who read this book do so because they believe in the promise of anomaly detection, but are confused by the furious debates in thought-leadership circles surrounding the topic. We intend this book to help demystify the topic and clarify some of the fundamental choices that have to be made in constructing anomaly detection mechanisms. We want readers to understand why some approaches to anomaly detection work better than others in some situations, and why a better solution for some challenges may be within reach after all.

This book is not intended to be a comprehensive source for all information on the subject. That book would be 1000 pages long and would be incomplete at that. It is also not intended to be a step-by-step ...