Practical Machine Learning: A New Look at Anomaly Detection

Chapter 1. Looking Toward the Future

Everyone loves a mystery, and at the heart of it, that’s what anomaly detection is—spotting the unusual, catching the fraud, discovering the strange activity. Anomaly detection has a wide range of useful applications, from banking security to natural sciences to medicine to marketing. Anomaly detection carried out by a machine-learning program is actually a form of artificial intelligence. With the ever-increasing volume of data and the new types of data, such as sensor data from an increasingly large variety of objects that needs to be considered, it’s no surprise that there also is a growing interest in being able to handle more decisions automatically via machine-learning applications. But in the case of anomaly detection, at least some of the appeal is the excitement of the chase itself.

Figure 1-1. Finding anomalies is the detective work of machine learning.

When are anomaly-detection methods a good choice? Unlike fictional detective stories, in anomaly detection, you may not have a clear suspect to search for, and you may not even know what the “crime” is. In fact, one way to think about when to turn to anomaly detection is this: Anomaly detection is about finding what you don’t know to look for.

You are searching for anomalies, but you don’t know what their characteristics will be. If you did, you could use a different form of machine learning, ...