Deep Learning with Python: A Hands-on Introduction

© Nikhil Ketkar 2017

Nikhil Ketkar, Deep Learning with Python, https://doi.org/10.1007/978-1-4842-2766-4_13

13. Regularization Techniques

Nikhil Ketkar¹ 

(1)Bangalore, Karnataka, India

In this chapter we will cover three regularization techniques commonly used in Deep Learning, namely, early stopping, norm penalties, and dropout. The reader is advised to refer to Chapter 2 introducing the basics of machine learning, specifically to revisit the notions of model capacity, overfitting, and underfitting.

Model Capacity , Overfitting , and Underfitting

Let us briefly revisit the notions of model capacity, overfitting, and underfitting. We will use the previous example (from Chapter 2) of fitting a regression model. We have data of the form where and ...