Similarly:
I
G
Salty =
2
3
1−
2
3
+
1
2
1−
1
2
=
2
9
+
1
4
=
17
36
What this means is that the GINI impurity for Salty is higher than the GINI impurity
for Sweet. Intuitively while creating a decision tree we would want to choose Sweet as
a split point first, since it will create less impurity in the tree.
Variance Reduction
Variance reduction is used primarily in continuous decision trees. Conceptually var‐
iance reduction aims to reduce the dispersion of the classification. While it doesn’t
apply to classification problems such as whether mushrooms are edible or not, it ...