Wavelet Neural Networks: With Applications in Financial Engineering, Chaos, and Classification

4 Model Selection: Selecting the Architecture of the Network

In this chapter we describe the model selection procedure. One of the most crucial steps is to identify the correct topology of the network. A desired wavelet network architecture should contain as few hidden units as necessary; at the same time it should explain as much variability of the training data as possible. A network with fewer hidden units than needed would not be able to learn the underlying function; selecting more hidden units than needed would result in an overfitted model. Therefore, it is essential to derive an algorithm to select the appropriate wavelet network model for a given problem.

The simplest way to select the optimal number of hidden units—in other words, the architecture of the wavelet network—is by trial and error, a method called exhaustive search. To do so, the training patterns must be split into a training sample and a validation sample. This method suggests that the optimum number of wavelons is given by the structure of the wavelet network that gives the minimum error on the validation set. This method is very simple but also very time consuming, and the information in the validation sample is not utilized for better training of the wavelet network.

In early stopping a fixed and large number of hidden units is used in construction of the network. The weights are allowed to change during the training phase. These free parameters are growing during the training phase. In early stopping ...