Hands-On Machine Learning for Algorithmic Trading

Cross-validation results across models

When comparing average cross-validation AUC across the four test runs with the three libraries, we find that CatBoost produces a slightly higher AUC score for the top-performing model, while also producing the widest dispersion of outcomes, as shown in the following graph:

The top-performing CatBoost model uses the following parameters (see notebook for detail):

• max_depth of 12 and max_bin of 128
• max_ctr_complexity of 2, which limits the number of combinations of categorical features
• one_hot_max_size of 2, which excludes binary features from the assignment of numerical variables
• random_strength different ...