Hands-On Java Deep Learning for Computer Vision

Deep network performance

In theory, it's clear that the more layers we add to the neuron network, the better it is. This is the case with the green line, as shown in the following graph. As soon as we add more layers, we'll see the error rate go down, ideally to zero. Unfortunately, in practice, we see this happening only partially. It's true that the error rate goes down once we add more layers, but only until a certain point. After this point, adding more layers will basically increase the error rate:

By adding many layers, we add a lot of weights and parameters, making it difficult for the neural network to figure all these weights out. ...