Mastering Machine Learning with scikit-learn - Second Edition

Regularization

Regularization is a collection of techniques that can be used to prevent overfitting. Regularization adds information, often in the form of a penalty against complexity, to a problem. Occam's razor states that the hypothesis with the fewest assumptions is best. Accordingly, regularization attempts to find the simplest model that explains the data.

scikit-learn provides several regularized linear regression models. Ridge regression, also known as Tikhonov regularization, penalizes model parameters that become too large. Ridge regression modifies the RSS cost function by adding the L² norm of the coefficients, as follows:

Lambda ...