Deep Learning Essentials

Backpropagation through time

In feedforward networks, backpropagation (BP) starts with calculating the final error at the output layer and then moving backward towards the inputs, layer by layer. At each step, it calculates the partial derivatives of the error versus the weights . Then through the optimization approach (for example, gradient descent), those derivatives are used to adjust the weights up or down in the direction that reduces the error.

Similarly in recurrent networks, after the unroll of the networks through time, BP can be thought of as an extension over the time dimension, this is called backpropagation through time or BPTT ...