We can increase the depth of recurrent neural networks by stacking them on top of each other. Essentially, we will be taking the target outputs and feeding them into another network.
To get an idea of how this might work for just two layers, see the following diagram:
Figure 5: In the preceding diagram, we have extended one-layer RNNs so that they have two layers. For the original one-layer versions, see the diagrams in the introduction to the previous chapter. The left architecture illustrates a way of using a multi-layered RNN to predict one output from a sequence of outputs. The right architecture shows a way to use a multi-layered ...
