The Regularization Cookbook

7

Deep Learning Regularization

In this chapter, we will cover several tricks and techniques to regularize neural networks. We will reuse the L2 regularization technique, as we did in linear models, for example. But there are other techniques not yet presented in this book, such as early stopping and dropout, which will be covered in this chapter.

In this chapter, we’ll look at the following recipes:

• Regularizing a neural network with L2 regularization
• Regularizing a neural network with early stopping
• Regularization with network architecture
• Regularizing with dropout

Technical requirements

In this chapter, we will train neural networks on various tasks. This will require us to use the following libraries:

• NumPy
• Scikit-learn
• Matplotlib
• PyTorch ...