The RNN architecture has been extended in many ways to accommodate the extra needs in certain problems and to overcome the shortcomings of simple RNN models. We list some of the major extensions to the RNN architecture below.
- Bidirectional RNN (BRNN) is used when the output depends on both the previous and future elements of a sequence. BRNN is implemented by stacking two RNNs, known as forward and backward Layer, and the output is the result of the hidden state of both the RNNs. In the forward layer, the memory state h flows from time step t to time step t+1 and in the backward layer the memory state flows from time step t to time step t-1. Both the layers take same input xt at time step t, but they jointly produce the output ...
