Hands-On Machine Learning for Algorithmic Trading

Beware of backtest overfitting

We covered the risks of false discoveries due to overfitting to historical data repeatedly throughout the book. Chapter 5, Strategy Evaluation lays out the main drivers and potential remedies. The low noise-to-signal ratio and relatively small datasets (compared to a web-scale image or text data) make this challenge particularly serious in the trading domain. Awareness is critical because the ease of access to data and tools to apply ML increase the risks exponentially.

There is no escape because there is no method of prevention. However, we presented methods to adjust backtest metrics to account for repeated trials such as the deflated Sharpe ratio. When working towards a live trading strategy, stage paper-trading ...