16
Testing, Evaluating, and Trading a Mechanical Trading System
The previous chapter showed you how to design a trading system. This chapter shows you how to properly test and evaluate a mechanical trading system. Testing and evaluating a trading system involves three major issues:
- How well can the system be expected to perform, and what are the risks?
- How likely is it that the system will continue to work in the future?
- Can we predict when and if the system's performance will either temporarily degrade or fail completely?
Addressing these issues requires a deep understanding of the trading system being evaluated—not only how the system generates its trading signals but a statistical analysis of the system via the development set, the testing set, and an out-of-sample set. Prediction of future system performance during live trading requires a statistical analysis that compares the system performance for both live trading and historical test periods.
THE STEPS FOR TESTING AND EVALUATING A TRADING SYSTEM
Table 16.1 gives a general outline for both evaluating and trading a mechanical trading system. This section discusses each of these steps in more detail.
Historical System Performance and Data Collection
In this stage of our analysis, we need to collect statistics about system performance. Data are collected on the system over the development, testing, and out-of-sample periods. ...
Get Cybernetic Trading Strategies: Developing a Profitable Trading System with State-of-the-Art Technologies now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
