Deep Learning Quick Reference

Stateful versus stateless LSTMs

Earlier in the chapter, we talked about an RNN's ability to maintain state, or memory, across time steps.

When using Keras, LSTMs can be configured in two ways, stateful and stateless.

The stateless configuration is the default. When you use a stateless LSTM configuration, LSTM cell memory is reset every batch. This makes batch size a very important consideration. Stateless works best when the the sequences you're learning aren't dependent on one another. Sentence-level prediction of a next word might be a good example of when to use stateless.

The stateful configuration resets LSTM cell memory every epoch. This configuration is most commonly used when each sequence in the training set depends on the sequence ...