Apache Spark Deep Learning Cookbook

How it works...

Since neural networks use a gradient-based optimization technique to learn features present in data, it is essential that these gradients are preserved in order for the network to calculate an error based on the change in gradients. This section will describe how the exploding gradient problem occurs in recurrent neural networks:

• While using backpropagation, the network first calculates the error, which is nothing but the model output subtracted from the actual output squared (such as the square error).
• Using this error, the model then computes the change in error with respect to the change in weights (de/dw).
• The computed derivative multiplied by the learning rate  gives w, which is nothing but the change in weights. The ...