Hands-On Machine Learning for Algorithmic Trading

How to compute the gradient

The neural network comprises a set of nested functions as highlighted precedingly. Hence, the gradient of the loss function with respect to internal, hidden parameters is computed using the chain rule of calculus.

For scalar values, given the functions z = h(x) and y = o(h(x)) = o (z), we compute the derivative of y with respect to x using the chain rule, as follows:

For vectors, with z ∈ R^m and x ∈ Rⁿ so that the hidden layer, h, maps from Rⁿ to R^m and z = h(x) and y = o (z), we get the following:

We can express ...