Python Machine Learning By Example - Second Edition

Best practice 9 – deciding on whether or not to reduce dimensionality, and if so, how to do so

Feature selection and dimensionality are different in the sense that the former chooses features from the original data space, while the latter does so from a projected space from the original space. Dimensionality reduction has the following advantages that are similar to feature selection, as follows:

• Reducing the training time of prediction models, as redundant, or correlated features are merged into new ones
• Reducing overfitting for the same reason as previously
• Likely improving performance as prediction models will learn from data with less redundant or correlated features

Again, it is not guaranteed that dimensionality reduction will yield ...