Mastering Predictive Analytics with R - Second Edition

Feature selection

Our CPU model only came with six features. Often, we encounter real-world datasets that have a very large number of features arising from a diverse array of measurements. Alternatively, we may have to come up with a large number of features when we aren't really sure what features will be important in influencing our output variable. Moreover, we might have categorical variables with many possible levels from which we are forced to create a large number of new indicator variables, as we saw in Chapter 1, Gearing Up for Predictive Modeling. When our scenario involves a large number of features, we often find that our output only depends on a subset of these. Given k input features, there are 2^k distinct subsets that we can form, ...