Building Machine Learning Systems with Python - Third Edition

Detecting redundant features using filters

Filters try to clean up the feature space independent of any later-used machine learning method. They rely on statistical methods to find out which of the features are redundant or irrelevant. In the case of redundant features, a filter keeps only one per redundant feature group. Irrelevant features will simply be removed. In general, a filter works as depicted in the following workflow:

First, we filter out features that are redundant using statistics that only take into account the training data. We then check whether the remaining features are useful in classifying the label.