Mastering TensorFlow 1.x

LSTM network

When RNNs are trained over very long sequences of data, the gradients tend to become either very large or very small that they vanish to almost zero. Long Short-Term Memory (LSTM) networks address the vanishing/exploding gradient problem by adding gates for controlling the access to past information. LSTM concept was first introduced by Hochreiter and Schmidhuber in 1997. 

Read the following research paper on LSTM to get more information about origins of LSTM: S. Hochreiter and J. Schmidhuber, Long Short-Term Memory, Neural Comput., vol. 9, no. 8, pp. 1735–1780, 1997. http://www.bioinf.jku.at/publications/older/2604.pdf

In RNN, a single neural network layer of repeatedly used learning function φ is used, whereas, in LSTM, a ...