TensorFlow 1.x Deep Learning Cookbook

Long Short Term Memory (LSTM)

An LSTM network can control when to let the input enter the neuron, when to remember what has been learned in the previous time step, and when to let the output pass on to the next timestamp. All these decisions are self-tuned and only based on the input. At first glance, an LSTM looks difficult to understand but it is not. Let's use the following figure to explain how it works:

An example of an LSTM cell

First, we need a logistic function σ (see Chapter 2, Regression) to compute a value between 0 and 1 and control which piece of information flows through the LSTM gates. Remember that the logistic function is ...