Machine Learning Algorithms - Second Edition

ElasticNet

The last alternative is called ElasticNet and combines both Lasso and Ridge into a single model with two penalty factors: one proportional to L₁ norm and the other to L₂ norm. In this way, the resulting model will be sparse like a pure Lasso, but with the same regularization ability as provided by Ridge. The resulting loss function is as follows:

The ElasticNet class provides an implementation where the alpha parameter works in conjunction with l1_ratio (beta in the formula). The main peculiarity of ElasticNet is avoiding a selective exclusion of correlated features, thanks to the balanced action of the L₁ and L₂ norms.

In the following ...