R Deep Learning Projects

LSTM

For an LSTM, we compute a number of things:

• An input gate, i
• A forget gate, f
• An output gate, o
• A candidate internal state, g
• An internal memory of the unit, c
• A final internal state, s

These are defined by the following recursive equations:

Where:

The first three equations describe the input, forget, and output gates. They are called gates because they decide which fraction of the newly computed candidate state will be allowed through, and which fraction will be forgotten. Given the new observations, we might want to keep something ...