Chapter 9NEURAL NETWORKS
9.1 INTRODUCTION TO NEURAL NETWORKS
Neural networks represent an attempt at a very basic level to imitate the type of nonlinear learning that occurs in the networks of neurons found in nature, such as the human brain. As shown in Figure 9.1, a neuron from the human brain uses dendrites to gather inputs from other neurons and combines the input information, generating a nonlinear response (“firing”) when some threshold is reached, which it sends to other neurons using the axon. Figure 9.1 also shows an artificial neuron model used in most neural networks. The inputs (xi) are collected from upstream neurons (or the data set) and combined through a combination function such as summation (Σ), which is then input into a (usually nonlinear) activation function to produce an output response (y), which may then be channeled downstream to other neurons.
Figure 9.1 Real neuron and artificial neuron model.
The main benefit of neural networks is that they are quite robust for noisy, complicated, or nonlinear data, due to the nonlinear nature of the activation function. On the other hand, the main drawback of neural networks is that they are relatively opaque to human interpretation, as opposed to, say, decision trees.
9.2 THE NEURAL NETWORK STRUCTURE
Let us examine the simple neural network shown in Figure 9.2.
Figure 9.2 Simple example of a neural network. ...