R Deep Learning Essentials - Second Edition

L2 penalty in action

To see how the L2 penalty works, we can use the same simulated linear regression problem we used for the Ll penalty. To fit a ridge regression model, we use the glmnet() function from the glmnet package. As mentioned previously, this function can actually fit the L1 or the L2 penalties, and which one occurs is determined by the argument, alpha. When alpha = 1, it fits lasso, and when alpha = 0, it fits ridge regression. This time, we choose alpha = 0. Again, we evaluate a range of lambda options and tune this hyper-parameter automatically using cross-validation. This is accomplished by using the cv.glmnet() function. We plot the ridge regression object to see the error for a variety of lambda values:

m.ridge.cv <- cv.glmnet(X[1:100, ...