Recurrent neural networks (RNNs) are designed to process sequential data such as text and audio. Their biggest difference to CNNs is that the weights in the hidden layers (that is, certain functions) are used repeatedly on multiple inputs and the order of the inputs affects the final results of the functions. The typical design of an RNN can be seen in the following diagram:
Figure 10.1 Basic computational units of a recurrent neural network
As we can see, the most distinctive characteristic of an RNN unit is that the hidden state, , has an outgoing connection pointing to itself. This self-loop is where ...
