Hands-On Machine Learning for Algorithmic Trading

The output layer

The output layer compresses the three-dimensional hidden layer activations, H, back to two dimensions using a 3 x 2 weight matrix, W^o, and a two-dimensional bias vector, b^o, as follows:

The linear combination of the hidden layer outputs results in an N x 2 matrix, Z^o, as follows:

The output layer activations are computed by the softmax function, ς, which normalizes the Z^o to conform to the conventions used for discrete probability distributions, as follows:

We define a softmax function in Python as follows:

def softmax(z): ...