Supervised Machine Learning with Python

How do decision trees make decisions?

We will first address the objective before looking at the math. We will compute the information gain of a split to determine the best splitting point. If information gain is positive, that means we have learned something from that split, which might be the optimal point. If information gain is negative, it means we're actually going in the wrong direction. What we have done is created a non-informative split. Each split in the tree will select the point that maximizes information gain.

So, here's the setup:

A Gini impurity of 0 would be particularly pure. A higher impurity essentially means that a more ...