Advances in Financial Machine Learning

CHAPTER 14 Backtest Statistics

14.1 Motivation

In the previous chapters, we have studied three backtesting paradigms: First, historical simulations (the walk-forward method, Chapters 11 and 12). Second, scenario simulations (CV and CPCV methods, Chapter 12). Third, simulations on synthetic data (Chapter 13). Regardless of the backtesting paradigm you choose, you need to report the results according to a series of statistics that investors will use to compare and judge your strategy against competitors. In this chapter we will discuss some of the most commonly used performance evaluation statistics. Some of these statistics are included in the Global Investment Performance Standards (GIPS),¹ however a comprehensive analysis of performance requires metrics specific to the ML strategies under scrutiny.

14.2 Types of Backtest Statistics

Backtest statistics comprise metrics used by investors to assess and compare various investment strategies. They should help us uncover potentially problematic aspects of the strategy, such as substantial asymmetric risks or low capacity. Overall, they can be categorized into general characteristics, performance, runs/drawdowns, implementation shortfall, return/risk efficiency, classification scores, and attribution.

14.3 General Characteristics

The following statistics inform us about the general characteristics of the backtest:

• Time range: Time range specifies the start and end dates. The period used to test the strategy should be sufficiently ...