Deep Learning Essentials

Recurrent neural networks (RNN/LSTM)

In a convolutional neural network or typical feedforward network, the information flows through a series of mathematical operations performed on the nodes without a feedback loop or any consideration of the order of the signal. Therefore, they are not capable of handling inputs which come in a sequence.

However, in practice, we have a lot of sequential data such as sentences and time series data, which include text, genomes, handwriting, the spoken word, or numerical times-series data emanating from sensors, stock markets, and government agencies. It is not only the order that matters here. The next value in line often greatly depends on the past context (long or short). For example, to predict the next ...