R Statistics Cookbook

How it works...

Ridge is similar to ordinary least squares, with the obvious difference that there is a penalization term. The idea is to minimize the squared residuals, and at the same time have coefficients that are not big. Because the coefficients won't be as big as they would have been in the absence of the penalization term, the model won't be able to over-fit to the data. In the following equation, we have the Ridge minimization problem; note that the is a hyper-parameter that defines how much weight we want to place on the penalization. A large value implies that the penalization will dominate and the model will likely under-fit (not ...