Hands-On Deep Learning Algorithms with Python

Input gate

The input gate is responsible for deciding what information should be stored in the cell state. Let's consider the same example:

Harry is a good singer. He lives in New York. Zayn is also a good singer.

After the forget gate removes information from the cell state, the input gate decides what information it has to keep in the memory. Here, since the information about Harry is removed from the cell state by the forget gate, the input gate decides to update the cell state with information about Zayn.

Similar to the forget gate, the input gate is controlled by a sigmoid function that returns output in the range of 0 to 1. If it returns 1, then the particular information will be stored/updated to the cell state, and if it returns ...