Backpropagation takes place once feed forward is completed. It stands for backward propagation of errors. In the case of neural networks, this step begins to compute the gradient of error function (loss function) with respect to the weights. One can wonder why the term back is associated with it. It's due to gradient computation that starts backwards through the network. In this, the gradient of the final layer of weights gets calculated first and the the weights of the first layer are calculated last.

Backpropagation needs three elements:

  • Dataset: A dataset that consists of pairs of input-output  where  is the input and 

Get Machine Learning Quick Reference now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.