Discovering Knowledge in Data: An Introduction to Data Mining, 2nd Edition

Chapter 9Neural Networks

1. 9.1 Input and Output Encoding
2. 9.2 Neural Networks for Estimation and Prediction
3. 9.3 Simple Example of a Neural Network
4. 9.4 Sigmoid Activation Function
5. 9.5 Back-Propagation
6. 9.6 Termination Criteria
7. 9.7 Learning Rate
8. 9.8 Momentum Term
9. 9.9 Sensitivity Analysis
10. 9.10 Application of Neural Network Modeling
    1. The R Zone
    2. References
    3. Exercises
    4. Hands-On Analysis

The inspiration for neural networks was the recognition that complex learning systems in animal brains consisted of closely interconnected sets of neurons. Although a particular neuron may be relatively simple in structure, dense networks of interconnected neurons could perform complex learning tasks such as classification and pattern recognition. The human brain, for example, contains approximately 10¹¹ neurons, each connected on average to 10,000 other neurons, making a total of 1,000,000,000,000,000 = 10¹⁵ synaptic connections. Artificial neural networks (hereafter, neural networks) represent an attempt at a very basic level to imitate the type of nonlinear learning that occurs in the networks of neurons found in nature.

As shown in Figure 9.1, a real neuron uses dendrites to gather inputs from other neurons and combines the input information, generating a nonlinear response (“firing”) when some threshold is reached, which it sends to other neurons using the axon. Figure 9.1 also shows an artificial neuron model used in most neural networks. The inputs (x_i) are collected from upstream neurons (or the ...