Arun Kejariwal (Independent) and Ira Cohen (Anodot) share a novel two-step approach for building more reliable prediction models by integrating anomalies in them. The first step uses anomaly detection algorithms to discover anomalies in a time series in the training data. In the second, multiple prediction models, including time series models and deep networks, are trained, enriching the training data with the information about the anomalies discovered in the first step.
Anomaly detection for individual time series is a necessary but insufficient step due to the fact that anomaly detection over a set of live data streams may result in anomaly fatigue, thereby limiting effective decision making. One way to address the above is to carry out anomaly detection in a multidimensional space. However, this is typically very expensive computationally and hence not suitable for live data streams. Another approach is to carry out anomaly detection on individual data streams and then leverage correlation analysis to minimize false positives, which in turn helps in surfacing actionable insights faster.
They then walk you through marrying correlation analysis with anomaly detection, discuss how the topics are intertwined, and detail the challenges you may encounter based on production data. They also showcase how deep learning can be leveraged to learn nonlinear correlation, which in turn can be used to further contain the false positive rate of an anomaly detection system.
This session was recorded at the 2019 O'Reilly Artificial Intelligence Conference in New York.
- Title: Deep learning for time series data
- Release date: October 2019
- Publisher(s): O'Reilly Media, Inc.
- ISBN: 0636920339618
You might also like
Deep Learning from Scratch
With the resurgence of neural networks in the 2010s, deep learning has become essential for machine …
Designing Data-Intensive Applications
Data is at the center of many challenges in system design today. Difficult issues need to …
40 Algorithms Every Programmer Should Know
Learn algorithms for solving classic computer science problems with this concise guide covering everything from fundamental …
Practical Statistics for Data Scientists, 2nd Edition
Statistical methods are a key part of data science, yet few data scientists have formal statistical …