Hands-On Deep Learning Algorithms with Python

Update gate

The update gate helps to decide what information from the previous time step, , can be taken forward to the next time step, . It is basically a combination of an input gate and a forget gate, which we learned about in LSTM cells. Similar to the gates about the LSTM cell, the update gate is also regulated by the sigmoid function.

The update gate, , at time step is expressed as follows:

Here, the following applies:

• is the input-to-hidden ...