Mastering Machine Learning with scikit-learn - Second Edition

Gini impurity

In the previous section, we built a decision tree by creating nodes that produced the greatest information gain. Another common heuristic for learning decision trees is Gini impurity, which measures the proportions of classes in a set. Gini impurity is given by the following equation, where j is the number of classes, t is the subset of instances for the node, and P(i|t) is the probability of selecting an element of class i from the node's subset:

Intuitively, Gini impurity is 0 when all the elements of the set are the same class, as the probability of selecting an element of that class is equal to 1. Like entropy, Gini impurity ...