Data Preprocessing and Postprocessing


Preprocessing and postprocessing data involve transforming data to make relationships more obvious or to extract information from raw data. Preprocessing refers to transforming raw data into a form that makes it easier for a modeling method, such as a neural network or machine induction, to find hidden relationships in the data, which can be used for forecasting. Postprocessing is the act of processing results from a model in order to extract all of the knowledge the model has learned. To illustrate, many neural-network-based models require postprocessing to make the forecasts useful. For example, we might find that the model is more accurate when the output of the neural network is above a given threshold.


There are many steps in developing good preprocessing. These steps are shown in Table 17.1.

Now that we have overviewed the steps involved in preprocessing, let us discuss them in more detail.


  1. Select the modeling method you are going to use.
  2. Decide on the half-life of the model you want to build. For example, you might want to develop a model that you must retrain every 20 days (or every 3 or 6 months, or once a year). This issue is not as easy to resolve as picking the longest period, because the longer the life you want your model to have, the more ...

Get Cybernetic Trading Strategies: Developing a Profitable Trading System with State-of-the-Art Technologies now with the O’Reilly learning platform.

O’Reilly members experience live online training, plus books, videos, and digital content from nearly 200 publishers.