Mastering Machine Learning Algorithms

LSTM

This model (which represents the state-of-the-art recurrent cell in many fields) was proposed in 1997 by Hochreiter and Schmidhuber (in Long Short-Term Memory, Hochreiter S., Schmidhuber J., Neural Computation, Vol. 9, 11/1997) with the emblematic name long-short-term memory  (LSTM). As the name suggests, the idea is to create a more complex artificial recurrent neuron that can be plugged into larger networks and trained without the risk of vanishing and, of course, exploding gradients. One of the key elements of classic recurrent networks is that they are focused on learning, but not on selectively forgetting. This ability is indeed necessary for optimizing the memory in order to remember what is really important and removing all those ...