Artificial Intelligence for Asset Management and Investment

Chapter 6Evaluation

GETTING THE MODEL RUNNING IS ONE THING; ensuring that it works, another. The term “works” implies two things:

The model works: This means the algorithm is efficient, generalizes well, and is accurate. Efficiency implies it does not take forever to run. Generalizes well means that it performs when introduced to data beyond the data on which it has been trained. Accurate means that it performs with high accuracy.

The solution works: This is specific to investment solutions. This includes backtesting, which implies that the strategy for which the model was developed is tested for performance.

In this chapter, we will introduce the organization, work process, and some best practices in evaluation. The discussion in this chapter mostly relates to classification methods where labeled data is used to train an algorithm.

WHO PERFORMS THE EVALUATION?

The evaluation function is headed by an executive with strong background in machine learning and finance. The organization reporting to the executive is composed of two different teams: (1) the investment strategy evaluation team, which has specialists who understand investment, asset classes, and instruments; and (2) the business/function team that works for enterprise and function-specific systems (Figure 6.1). The investment strategy team performs evaluation for investment strategies and is segmented into algorithm evaluation and backtesting evaluation. The business and function-specific team evaluates models for ...