R Deep Learning Cookbook

Setting up a deep RNN model

The RNN architecture is composed of input, hidden, and output layers. A RNN network can be made deep by decomposing the hidden layer into multiple groups or by adding computational nodes within RNN architecture such as including model computation such as multilayer perceptron for micro learning. The computational nodes can be added between input-hidden, hidden-hidden, and hidden-output connection. An example of a multilayer deep RNN model is shown in the following figure:

An example of two-layer Deep Recurrent Neural Network architecture