Hands-On Artificial Intelligence for Beginners

LSTM

LSTMs are a special form of RNN that excel at learning long-term dependencies. Developed by Hochreiter and Schmidhuber in 1997, LSTMs have several different layers that information passes through to help them keep what is important and jettison the rest. Unlike vanilla recurrent networks, the LSTM has not one but two states; the standard hidden state that we've been representing as h_t, as well as a state that is specific to the LSTM cell called the cell state, which we will denote with c_t. These LSTM states are able to update or adjust these states with gating mechanisms. These gates help to control the processing of information through the cell, and consist of an activation function and a basic point wise operation, such as vector multiplication. ...