Advanced Machine Learning with R

Ridge regression

Let's begin by exploring what ridge regression is and what it can and can't do for you. With ridge regression, the normalization term is the sum of the squared weights, referred to as an L2-norm. Our model is trying to minimize RSS + λ(sum Bj²). As lambda increases, the coefficients shrinks toward zero but never become zero. The benefit may be an improved predictive accuracy but, as it doesn't zero out the weights for any of your features, it could lead to issues in the model's interpretation and communication. To help with this problem, we can turn to LASSO.