Neural Network Projects with Python

Output gate

Finally, the output gate controls how much information is to be retained in the hidden state. The output gate is highlighted in the following diagram:

First, we concatenate the Previous Hidden State (h_t−1) and the current Input (x_t), and pass it through a sigmoid function. Then, we take the current cell state (C_t) and pass it through a tanh function. Finally, we take the multiplication of the two, which is passed to the next repeating unit as the hidden state (h_t). This process is summarized by the following equation: