Hands-On Machine Learning for Algorithmic Trading

The challenges of high-dimensional data

We have seen how the key challenge of supervised learning is to generalize from training data to new samples. Generalization becomes exponentially more difficult as the dimensionality of the data increases. We encountered the root causes of these difficulties when we covered the curse of dimensionality in Chapter 12, Unsupervised Learning.

One aspect of this curse is that volume grows exponentially with the number of dimensions: the volume of a hypercube with edge length 10 increases from 10³ to 10⁴ as the number of dimensions goes from three to four. Consequently, the number of data points required to maintain a given density of observations also grows exponentially.

Moreover, functional relationships ...