Mastering Machine Learning with scikit-learn - Second Edition

Selecting the questions

Like Twenty Questions, the decision tree will estimate the value of the response variable by testing the values of a sequence offeatures. Which featureshould be tested first? Intuitively, a test that produces subsets that contain all cats or all dogs is better than a test that produces subsets that contain both cats and dogs. If the members of a subset are of different classes, we are still uncertain about how to classify the instance. We should also avoid creating tests that separate only a single cat or dog from the others; such tests are analogous to asking specific questions in the first few rounds of Twenty Questions. These tests can infrequently classify an instance and will not likely reduce our uncertainty. ...