Our original node always starts at the root of the tree. In the preceding diagram, the first node merely starts with a description of how the population breaks down by gender. In this case, females make up the majority class at 58% of the population. The goal in decision of tree algorithms is to always grow a tree and improve predictions. So, at this point, we want to do better than 58% prediction for gender.
As we grow the tree, two questions will always need to be taken into consideration:
- Which variable will best improve the prediction for each of the classes of the target variable at that point?
- For that variable, what is the best split point that will segment a categorical variable into two parts, or divide a continuous ...