Hands-On Artificial Intelligence for Cybersecurity

Anomaly detection assumptions and challenges

From a methodological point of view, there is no doubt that outliers represent a problem for learning algorithms, as they constitute a disturbing element in the identification of a descriptive model, starting from training data.

When dealing with an anomalous value, how should the algorithm behave? Should it take into account the determination of the model or should it discard it, considering it as an estimation error? Or do the outliers represent novelties in the dataset that testify to real changes in the phenomenon being under scrutiny? To answer these questions, we need to investigate the most probable origin of the outliers.

In some cases, the outliers are a combination of uncommon values, ...