Pattern Recognition, 4th Edition

The Backpropagation Algorithm

• Initialization: Initialize all the weights with small random values from a pseudorandom sequence generator.
• Forward computations: For each of the training feature vectors x(i), i = 1, 2,…, N, compute all the v^rj(i), y^rj(i) = f(v^rj(i)), j = 1, 2,…, k^r, r = 1, 2,…, L, from (4.7). Compute the cost function for the current estimate of weights from (4.5) and (4.14).
• Backward computations: For each i = 1, 2,…, N and j = 1, 2,…, k^L compute δ^Lj(i) from (4.15) and in the sequel compute δ^r−1j(i) from (4.22) and (4.23) for r = L, L − 1,…, 2, and j = 1, 2,…, k^r
• Update the weights: For r = 1, 2,…, L and j = 1, 2,…, k^r

Remarks ...