Image Analysis, Classification and Change Detection in Remote Sensing, 4th Edition

Β

Efficient Neural Network Training Algorithms

The standard backpropagation algorithm introduced in Chapter 6 is notoriously slow to converge. In this appendix we will develop two additional training algorithms for the two-layer, feed-forward neural network of Figure 6.10. The first of these, the scaled conjugate gradient, makes use of the second derivatives of the cost function with respect to the synaptic weights, i.e., of the Hessian matrix. The second, the extended Kalman filter method, takes advantage of the statistical properties of the weight parameters themselves. Both techniques are considerably more efficient than backpropagation.

B.1    The Hessian matrix

We begin with a detailed discussion of the Hessian matrix and how to calculate ...