Python Data Science Essentials - Third Edition

Stability and L1-based selection

Though effective, recursive elimination is actually a step-by-step algorithm that bases its choices on sequences of single evaluations. While pruning, it opts for certain selections, potentially excluding many others. That's a good way to reduce a particularly challenging and time-consuming problem, such as an exhaustive search among possible sets, into a more manageable one. Anyway, there's another way to solve the problem, which is by using all the variables at hand conjointly. Some algorithms use regularization to limit the weight of the coefficients, thus preventing overfitting and the selection of the most relevant variables without losing predictive power. In particular, the regularization L1 (the lasso) ...