A recurrent neural network (RNN) can be used to train a model with a temporal dependence, such as language. It can be used to train any kind of sequence data. The output of a neuron is fed back into itself in an RNN. An RNN, when unrolled, looks as follows:
Due to its nature of taking input from previous time-steps, the data from a sequence that occurred long ago is lost.