Python: Real World Machine Learning

Feature selection by regularization

In a batch context, it is common to operate feature selection by the following:

• A preliminary filtering based on completeness (incidence of missing values), variance, and high multicollinearity between variables in order to have a cleaner dataset of relevant and operable features.
• Another initial filtering based on the univariate association (chi-squared test, F-value, and simple linear regression) between the features and response variable in order to immediately remove the features that are of no use for the predictive task because they are little or not related to the response.
• During modeling, a recursive approach inserting and/or excluding features on the basis of their capability to improve the predictive ...