Hands-On Deep Learning Algorithms with Python

Gated recurrent units

So far, we have learned about how the LSTM cell uses different gates and how it solves the vanishing gradient problem of the RNN. But, as you may have noticed, the LSTM cell has too many parameters due to the presence of many gates and states.

Thus, while backpropagating the LSTM network, we need to update a lot of parameters in every iteration. This increases our training time. So, we introduce the Gated Recurrent Units (GRU) cell, which acts as a simplified version of the LSTM cell. Unlike the LSTM cell, the GRU cell has only two gates and one hidden state.

An RNN with a GRU cell is shown in the following diagram: