Advanced Deep Learning with Python

CNN regularization

As we discussed in Chapter 1, The Nuts and Bolts of Neural Networks, a NN can approximate any function. But with great power comes great responsibility. The NN may learn to approximate the noise of the target function rather than its useful components. For example, imagine that we are training a NN to classify whether an image contains a car or not, but for some reason, the training set contains mostly red cars. It may turn out that the NN will associate the color red with the car, rather than its shape. Now, if the network sees a green car in inference mode, it may not recognize it as such because the color doesn't match. This problem is referred to as overfitting and it is central in machine learning (and even more so ...