Practical Time Series Analysis

Deep recurrent neural networks

The power of deep learning comes with stacking multiple computational layers on top of each other. In case of MLPs, multiple hidden layers are placed against each other. We can make deep RNNs by stacking multiple RNNs on top of each other. In a deep RNN, the input sequence for a recurrent layer is the output sequence of the previous recurrent layer. The final prediction is taken from the last timestep of the final RNN layer. The following figure illustrates a deep RNN:

Figure 5.18: Deep recurrent neural network with p timesteps

We can create deep bi-directional RNN as well by stacking multiple bi-directional ...