Machine Learning for OpenCV 4 - Second Edition

Controlling the complexity of decision trees

If you continue to grow a tree until all leaves are pure, you will typically arrive at a tree that is too complex to interpret. The presence of pure leaves means that the tree is 100 percent correct on the training data, as was the case with our tree shown earlier. As a result, the tree is likely to perform very poorly on the test dataset, as was the case with our tree shown earlier. We say the tree overfits the training data.

There are two common ways to avoid overfitting:

• Pre-pruning: This is the process of stopping the creation of the tree early.
• Post-pruning (or just pruning): This is the process of first building the tree but then removing or collapsing nodes that contain only a little information. ...