We just learned how the update and reset gates work, but how do these gates help in updating the hidden state? That is, how do you add new information to the hidden state and how do you remove unwanted information from the hidden state with the help of the reset and update gates?
First, we will see how to add new information to the hidden state.
We create a new state called the content state, , for holding the information. We know that the reset gate is used to remove information that is not required. So, using the reset gate, we create a content state, , that holds only the required information.
The content state, , at ...