Deep Learning with R Cookbook

Text generation using LSTMs

Recurrent neural networks face difficulties in carrying information properly, especially when there are long order dependencies between layers in large networks. Long-short term memory networks, generally referred to as LSTM networks, are an extension of RNNs that are capable of learning long-term dependencies and are widely used in deep learning to avoid the vanishing gradient problem that's faced by RNNs. LSTMs combat vanishing gradients through a gating mechanism and are able to remove or add information to the cell state. This cell state is carefully regulated by the gates, which control the information that's passed through them. LSTMs have three kinds of gates: input, output, and forget. The forget gate controls ...