Hands-On Data Science for Marketing

Growing decision trees

When we are growing decision trees, the trees need to come up with a logic to split a node into child nodes. There are two main methods that are commonly used for splitting the data: Gini impurity and entropy information gain. Simply put, Gini impurity measures how impure a partition is, and entropy information gain measures how much information it gains from splitting the data with the criteria being tested.

Let's take a quick look at the equation to compute the Gini impurity measure:

Here, c stands for the class labels, and P_i stands for the probability of a record with the class label i being chosen. By subtracting ...