Hands-On Deep Learning Algorithms with Python

Generating Song Lyrics Using RNN

In a normal feedforward neural network, each input is independent of other input. But with a sequential dataset, we need to know about the past input to make a prediction. A sequence is an ordered set of items. For instance, a sentence is a sequence of words. Let's suppose that we want to predict the next word in a sentence; to do so, we need to remember the previous words. A normal feedforward neural network cannot predict the correct next word, as it will not remember the previous words of the sentence. Under such circumstances (in which we need to remember the previous input), to make predictions, we use recurrent neural networks (RNNs).

In this chapter, we will describe how an RNN is used to model sequential ...