Tree-Based Machine Learning Methods in SAS Viya

Chapter 6: Ensemble of Trees: Bagging, Boosting, and Forest

“Small changes in the training data can cause large changes in the topology of the tree. However, the overall performance of the tree remains stable.”

– Breiman et al. (1984)

Instability in a Decision Tree

Decision trees are among the simplest, yet most powerful, forms of models, both conceptually and in their interpretation, with the entire data set of observations being recursively partitioned into subsets along different branches so that similar observations get grouped together at the terminal leaves of the decision tree. However, the methods of constructing decision trees from data suffer from a major problem of instability. The symptoms of instability include variations in the ...