Hands-On Convolutional Neural Networks with TensorFlow

L2 and L1 regularization

The first way that we will look at to create a more robust model is to use L1 or L2 regularization. These are by far the most common methods of regularization. The basic idea is that during training of our model, we actively try to impose some constraint on the values of the model weights using either the L1 or L2 norms of those weights.

We do this by adding an extra term to whatever loss function we are using. For L1 regularization, the term we add is , and for L2 regularization, the term we add is . In the preceding ...