Finding Alphas, 2nd Edition

16Machine Learning in Alpha Research

By Michael Kozlov

INTRODUCTION

Over the past several decades, machine learning has become a common tool for almost any task that requires information extraction from large datasets. In alpha research, several common problems can be solved using a machine learning methodology:

• Regression problems, in which Y and X are quantitative variables and Y is inferred by a function .
• Classification problems, in which Y is a qualitative variable and inferred from a quantitative variable X.
• Cauterization problems, in which a quantitative variable X is observed and classified into groups with similar features.

In this chapter, we will introduce the most common techniques used to address these problems.

In alpha research, it is not important to describe perfectly what has happened in the past, but it is important to be as precise as possible in predicting the future. Therefore, we are faced with the following dilemma: an overly complex model may enable perfect calibration but lead to overfitting and a poor quality of prediction, whereas an overly simplistic model that fits the sample data very poorly has no chance of predicting future behavior more accurately.

MACHINE LEARNING METHODS

Historically, the field of machine learning received a boost after World War II, when humanity faced a pressing need to analyze a lot of information and make a lot of correct ...