Hands-On Machine Learning for Algorithmic Trading

Universal function approximation

The Universal Approximation Theorem formalizes the powerful ability of neural networks to capture arbitrary relationships between input and output data. In 1989, George Cybenko showed that neural networks with a single layer of neurons connecting input and output using nonlinear, sigmoid activation functions are generally able to represent any continuous function on a closed and bounded subset of Rⁿ.

Kurt Hornik showed in 1991 that it is not the specific shape of the activation function but rather the multi-layered architecture that enables the hierarchical feature representation that allows neural networks to approximate universal functions.

However, the theorem also does not specify the network architecture ...