Deep Learning By Example

Step 1 – forward propagation

All weights in the network are randomly initialized. Let's consider a specific hidden layer node and call it V. Assume that the weights of the connections from the inputs to that node are w1, w2, and w3 (as shown).

The network then takes the first training samples as input (we know that for inputs 35 and 67, the probability of passing is 1):

• Input to the network = [35, 67]
• Desired output from the network (target) = [1, 0]

Then, output V from the node in consideration, which can be calculated as follows (f is an activation function such as sigmoid):

V = f (1*w1 + 35*w2 + 67*w3)

Similarly, outputs from the other node in the hidden layer are also calculated. The outputs of the two nodes in the hidden layer act ...