R Statistics Cookbook

How it works...

OLS works by minimizing the residual sum of squares, assuming a linear model. Both lasso and ridge assume the same idea, but with the difference of adding a penalization. Lasso adds an L1 one, whereas ridge adds an L2 one. This means that lasso results will be pushed towards zero (maybe exactly zero), and ridge will be pushed towards zero (but not exactly zero). Therefore, Lasso works almost as a variable selection tool: the irrelevant coefficients will receive a zero coefficient, thus being effectively dropped from the model. GLMNET is a mixture of both ridge and lasso, that is usually preferred by data scientists. The reason is that not only the amount of regularization, but also the mix of L2/L1 regularization can be tuned. ...