Practical Time Series Analysis

Gated Recurrent Units

GRU is simpler than LSTM and has only two internal gates, namely, the update gate (z_t) and the reset gate (r_t). The computations of the update and reset gates are as follows:

z_t = σ(W^zx_t + U^zs_t-1)

r_t = σ(W^rx_t + U^rs_t-1)

The state s_t of the timestep t is computed using the input x_t , state s_t-1 from the previous timestep, the update, and the reset gates:

The update being computed by a sigmoid function determines how much of the previous step's memory is to be retained in the current timestep. The reset gate controls how to combine the previous memory with the current step's input.

Compared to LSTM, which has three gates, ...