Python Deep Learning - Second Edition

Backpropagation in convolutional layers

In Chapter 2, Neural Networks, we talked about backpropagation in general, and for fully-connected layers in particular. In a fully-connected layer, an input neuron contributes to all output neurons. Because of this, when the gradient is routed back, all output neurons contribute back to the original neuron. In effect, we used the same operation of weighted sum in the forward and backward passes. The same rule applies for convolutional layers, where the neurons are locally-connected. In the Convolutional layers section, we observed how a neuron participates in the inputs of several output neurons. This is illustrated in the following diagram, where we can see a convolution operation with 3x3 filter. ...