Neural Networks with R

Forward and backpropagation

The processing from input layer to hidden layer(s) and then to the output layer is called forward propagation. The sum(input*weights)+bias is applied at each layer and then the activation function value is propagated to the next layer. The next layer can be another hidden layer or the output layer. The construction of neural networks uses large number of hidden layers to give rise to Deep Neural Network (DNN).

Once the output is arrived at, at the last layer (the output layer), we compute the error (the predicted output minus the original output). This error is required to correct the weights and biases used in forward propagation. Here is where the derivative function is used. The amount of weight that has to ...