Wavelet Neural Networks: With Applications in Financial Engineering, Chaos, and Classification

6 Model Adequacy: Determining a Network's Future Performance

In this chapter we present various metrics in order to assess a trained network. We are interested in measuring the predictive ability of a wavelet network in the context of a particular application. The evaluation of the model usually includes two clearly distinct, although related stages.

In the first stage, various metrics that quantify the accuracy of the predictions or the classifications made by the model are used and the model is evaluated based on these metrics. The term accuracy is a quantification of the “proximity” between the outputs of the wavelet network and the target values desired. The measurements of the precision are related to the error function that is minimized (or in some cases, the profit function that is maximized) during model specification of the wavelet network model. When an estimate of the model is done by minimizing the squared error function, the simplest example of such a measurement of accuracy is the mean squared error (MSE). The most common error criteria are the MSE, the root MSE, the normalized MSE, the sum of squared errors, the maximum absolute error, the mean absolute error, the (symmetric) mean absolute percentage error, and Theil's U index.

In addition, useful and immediate information can be provided by visual examination of a scatter plot of the network forecasts and the target values. Moreover, the statistical hypothesis testing of the values of the intercept and the slope ...