A key hyperparameter for decision trees is the maximum tree depth. We may decide that we want to make a maximum number of decisions before we predict. By default, the tree will be as deep as it can be. If we restrict the depth, we are trying to fight overfitting in our decision tree. The main pitfall to decision trees is that, in general, they have a propensity to overfit. The best way to combat this overfitting is to restrict the maximum depth of the tree. Smaller, simpler trees do less overfitting than deep, complex trees.
Let's see how we can restrict tree depth and decide what tree depth is appropriate. Although I have stated that we might want to restrict the depth, I haven't said what depth is best. For that, ...
