Feature Engineering Made Easy

A brief introduction to regularization

In linear models, regularization is a method for imposing additional constraints to a learning model, where the goal is to prevent overfitting and improve the generalization of the data. This is done by adding extra terms to the loss function being optimized, meaning that, while fitting, regularized linear models may severely diminish, or even destroy features along the way. There are two widely used regularization methods, called L1 and L2 regularization. Both regularization techniques rely on the L-p Norm, which is defined for a vector as being:

• L1 regularization, also known as lasso regularization, ...