Hands-On Deep Learning Algorithms with Python

Understanding the LSTM cell

What makes LSTM cells so special? How do LSTM cells achieve long-term dependency? How does it know what information to keep and what information to discard from the memory?

This is all achieved by special structures called gates. As shown in the following diagram, a typical LSTM cell consists of three special gates called the input gate, output gate, and forget gate:

These three gates are responsible for deciding what information to add, output, and forget from the memory. With these gates, an LSTM cell effectively keeps information in the memory only as long as required.

In an RNN cell, we used the hidden state, ...