Python Deep Learning - Second Edition

Regularization

We already know that overfitting is a central problem in machine learning (and even more so in deep networks). In this section, we'll discuss several ways to prevent it. Such techniques are collectively known as regularization. To quote Ian Goodfellow's Deep Learning book:

Regularization is any modification we make to a learning algorithm that is intended to reduce its generalization error but not its training error.