Computational Intelligence

208 Chapter Six—Neural Network Implementations

Before the weights can be updated, however, there must be something to update.

That is, each weight must be initialized to some value. You can’t just start out with all

weights equal to 0 (or all equal to any single number, for that matter), or the network

won’t be trainable. The reason can be seen by studying the weight update equations

presented next.

It is typical to initialize the weights in a back-propagation network to random

values between 0.3 and −0.3. Picking random numbers over some range makes

intuitive sense, and you can see how different weights go in different directions by

doing this. But ...