Machine Learning Quick Reference

Tree splitting

There are various algorithms that help when it comes to tree splitting, all of which take us to the leaf node. The decision tree takes all of the features (variables) that are available into account and selects the feature that would result in the most pure or most homogeneous split. The algorithm that's used to split the tree also depends on the target variable. Let's go through this, step by step:

1. Gini index: This says that if we select two items at random from a population, they must be from the same class. The probability for this event would turn out to be 1 if the population is totally pure. It only performs binary splits. Classification and regression trees (CARTs) make use of this kind of split.

The following formula ...