Python Deep Learning - Second Edition

Gated recurrent units

A gated recurrent unit (GRU) is a type of recurrent block that was introduced in 2014 by Kyunghyun Cho et al. (https://arxiv.org/abs/1406.1078, https://arxiv.org/abs/1412.3555), as an improvement over LSTM (see the following diagram). A GRU unit usually has similar or better performance to a LSTM, but it does so with fewer parameters and operations:

A GRU cell

Similar to the "classic" RNN, a GRU cell has a single hidden state, h_t. You can think of it as a combination of the hidden and cell states of an LSTM. The GRU cell has two gates:

• An update gate z_t, which is a combination of the input and forget LSTM gates. It decides ...