R Statistics Cookbook

How it works...

The mechanics here are the same as in the Ridge case, with the obvious difference that we use the L1 norm here:

There are some obvious differences with respect to the Ridge coefficients:

• For Lasso, we get 0 exactly for the irrelevant variables (this is a way of doing variable selection). But for Ridge, we get a smaller value (a compressed coefficient) but not exactly zero. Hence, Ridge can't be used for variable selection.
• Lasso coefficients are smaller than their Ridge counterparts (using the same lambda). In other words, the bias seems larger for Lasso.
• Ridge coefficients are more stable (measured as the height of the boxes ...