Introducing Machine Learning

Chapter 12. How to Make Complex Predictions and Decisions: Trees

The possible solutions to a given problem emerge as the leaves of a tree, each node representing a point of deliberation and decision.

—Niklaus Wirth(Creator of the Pascal programming language and winner of the Turing Award in 1984)

In the preceding chapter, we discussed regression as a way to infer the hidden relationship between dependent features and use it to build an effective prediction engine. Regression uses tools from statistics to perform its magic. Well, it’s not really magic—it’s a pure math matter of minimizing the values of a function that measures the error—but it sometimes really looks like magic in the sense that you get good predictions out of the nowhere of pure ...