Machine Learning Algorithms - Second Edition

Recurrent Neural Networks

A recurrent layer is made up of particular neurons that present recurrent connections so as to bind the state at time t to its previous values (in general, only one). This category of computational cells is particularly useful when it's necessary to capture the temporal dynamics of an input sequence. In many situations, in fact, we expect an output value that must be correlated with the history of the corresponding inputs. But an MLP, as well as the other models that we've discussed, are stateless. Therefore, their output is determined only by the current input. Recurrent Neural Networks (RNNs) overcome this problem by providing an internal memory that can capture short-term and long-term dependencies.

The most common ...