
Multilayer Feedforward Networks
85
needs to be done only on hidden-layer weights. For any given set of
hidden-layer weights, output-layer weights that deliver the global
minimum of the mean square error can be explicitly computed. This
is a significant savings, especially if there are many output neurons.
There is one potentially serious drawback to linear activation
functions. This concerns noise immunity. Although the squashing
functions in the hidden layer provide a fair degree of buffering, the
extra amount provided at the output layer can sometimes be valuable.
Also,
the problems of nonlinear output activation functions cited above
ar