Image Analysis, Classification and Change Detection in Remote Sensing, 4th Edition

Continuing, making use of Equation (6.35),

$\begin{array}{c}-{\delta }_k^o\left(v\right)={\displaystyle \sum _{k^{\prime }=1}^K-\frac{{\ell }_{k^{\prime }}\left(v\right)}{m_{k^{\prime }}\left(v\right)}{m}_k\left(v\right)\left({\delta }_{k{k}^{\prime }}-m_{k^{\prime }}\left(v\right)\right)}\\ =-{\ell }_k\left(v\right)+m_k\left(v\right){\displaystyle \sum _{k^{\prime }=1}^K{\ell }_{k^{\prime }}\left(v\right).}\end{array}$

But this last sum over the K components of the label ℓ(v) is just unity, and therefore we have

$\begin{array}{cc}-{\delta }_k^o\left(v\right)=-{\ell }_k\left(v\right)+m_k\left(v\right),& k=1\dots K,\end{array}$

which may be written as the K-component vector

${\delta }^o\left(v\right)=\ell \left(v\right)-m\left(v\right).$

(6.38)

From Equation (6.36), we can therefore express the third step in the back-propagation algorithm in the form of the matrix equation (see Exercise 6)

$W^o\left(v+1\right)=W^o\left(v\right)+\eta n\left(v\right){\delta }^o{\left(v\right)}^{\top }.$

(6.39)

Here W°(v+ 1) indicates the synaptic weight matrix after the update for vth training pair. Note that the second term on the right-hand side of Equation (6.39) is an outer product, yielding a matrix of dimension (L + 1) × K and so matching the dimension ...