- 9.1 Input and Output Encoding
- 9.2 Neural Networks for Estimation and Prediction
- 9.3 Simple Example of a Neural Network
- 9.4 Sigmoid Activation Function
- 9.5 Back-Propagation
- 9.6 Termination Criteria
- 9.7 Learning Rate
- 9.8 Momentum Term
- 9.9 Sensitivity Analysis
- 9.10 Application of Neural Network Modeling
The inspiration for neural networks was the recognition that complex learning systems in animal brains consisted of closely interconnected sets of neurons. Although a particular neuron may be relatively simple in structure, dense networks of interconnected neurons could perform complex learning tasks such as classification and pattern recognition. The human brain, for example, contains approximately 1011 neurons, each connected on average to 10,000 other neurons, making a total of 1,000,000,000,000,000 = 1015 synaptic connections. Artificial neural networks (hereafter, neural networks) represent an attempt at a very basic level to imitate the type of nonlinear learning that occurs in the networks of neurons found in nature.
As shown in Figure 9.1, a real neuron uses dendrites to gather inputs from other neurons and combines the input information, generating a nonlinear response (“firing”) when some threshold is reached, which it sends to other neurons using the axon. Figure 9.1 also shows an artificial neuron model used in most neural networks. The inputs (xi) are collected from upstream neurons (or the ...