Hands-On Deep Learning with Go

Backpropagation

This is a simple example. For complicated models with thousands, or even millions, of parameters across a number of layers, there are convolutional networks and we need to be more intelligent about how we propagate these updates back through our network. This is true for networks with a number of layers (increasing the number of parameters accordingly), with new research coming out that, in an extreme example, includes CNNs of 10,000 layers.

So, how can we go about this? The easiest way is to build your neural network out of functions for which we know the derivative. We can do this symbolically or on a more practical basis; if we build it out of functions where we know how to apply the function and where we know how to backpropagate ...