Machine Learning Algorithms - Second Edition

Feature importance in Random Forests

The concept of feature importance that we previously introduced can also be applied to Random Forests, computing the average over all of the trees in the forest:

We can easily test the evaluation of importance with a dummy dataset that contains 50 features with 20 noninformative elements:

nb_samples = 1000X, Y = make_classification(n_samples=nb_samples, n_features=50, n_informative=30, n_redundant=20, n_classes=2, n_clusters_per_class=5)

The importance of the first 50 features according to a Random Forest with 20 trees is plotted in the following graph:

Feature importance in a Random Forest

As expected, ...