In this chapter we will cover three regularization techniques commonly used in Deep Learning, namely, early stopping, norm penalties, and dropout. The reader is advised to refer to Chapter 2 introducing the basics of machine learning, specifically to revisit the notions of model capacity, overfitting, and underfitting.
Model Capacity , Overfitting , and Underfitting
Let us briefly revisit the notions of model capacity, overfitting, and underfitting. We will use the previous example (from Chapter 2) of fitting a regression model. We have data of the form where and ...
