Cells and gates in LTSM

LSTM networks consist of many connected LSTM cells, and each of them can be thought of as being composed of three important gates. Those gates decide whether the information from the past/present flow through.

The following figure shows a standard LSTM memory cell. In this cell, vector multiplication and vector addition are denoted by a round blue circle with a sign inside. Multiplying a vector with another vector that is in the range of [0,1] is called gating. Generating the vector in [0,1] can be thought of as a filtering process. is the previous (in time) cell state. is the updated cell state by the current input. ...

Get Deep Learning Essentials now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.