Hands-On Deep Learning with Apache Spark

RNN issues

The two major issues affecting RNNs are the Exploding Gradients and Vanishing Gradients. We talk about Exploding Gradients when an algorithm assigns, without a reason, a high importance to the model weights. But, the solution to this problem is easy, as this would require just truncating or compressing the gradients. We talk about Vanishing Gradients when the values of a gradient are so small that they cause a model to stop or take too long to learn. This is a major problem if compared with the Exploding Gradients, but it has now been solved through the LSTM (Long Short-Term Memory) neural networks. LSTMs are a special kind of RNN, capable of learning long-term dependencies, that were introduced by Sepp Hochreiter (https://en.wikipedia.org/wiki/Sepp_Hochreiter ...