TensorFlow 1.x Deep Learning Cookbook

Getting ready

The network is first presented with M training pairs (X, Y), with X as the input and Y the desired output. The input is propagated from input via the activation function g(h) through the hidden layers, up to the output layers. The output Y_hat is the output of the network, giving the error = Y- Y_hat.

The loss function J(W) is as follows:

Here, i varies over all the neurons of the output layer (1 to N). The change in weights W_ij, connecting i^th output layer neuron to j^th hidden layer neuron, can then be determined using the gradient of J(W) and employing the chain rule for differentiation:

Here, O_j is the output of the hidden ...