Inside the Black Box: The Simple Truth About Quantitative Trading

9.3. TESTING

The process of testing is central to research. At first glance, the most common version of this process looks fairly simple. First, build a model and train it on some subset of the data available (the in-sample period). Then test it on another subset of the data to see if it is profitable (the out-of-sample period). However, research is an activity that is fraught with peril. The researcher is constantly offered opportunities to forego rigor in favor of wishful thinking. In this section, we address some of the work and challenges inherent in the research process.

9.3.1. In-Sample Testing, a.k.a. Training

In quant trading, models are approximations of the world. They are used to predict the future using data as inputs. The first part of the testing process is to "train" a model by finding optimal parameters over an in-sample period. That sounds rather like a mouthful of marbles, so let's walk through it term by term.

Let's imagine that we want to test the idea that cheap stocks outperform expensive stocks. We even theorize that the metric we will use to define cheapness is the earnings yield (earnings/price), such that a higher earnings yield implies a cheaper stock. But what level of yield is sufficiently low to cause us to think that the stock will outperform? And what level of earnings yield is sufficiently high to imply that a stock is expensive and is likely to underperform? These levels are parameters. In general the parameters of a model are quantities that ...