Limitations of RNNs
Recurrent neural networks function just right when it comes to short-term dependencies. What this means is that, if there is just a single statement to be dealt with, a neural network operates fine. For example, if there is a sentence, India's capital is __, in this scenario we would invariably get the correct result as this is a universal statement and there is nothing like a context here. This statement has no dependency on the previous sentence and here, there is no previous sentence either.
Hence, the prediction would be India's capital is New Delhi.
Afterall, the vanilla RNN's does not understand the context behind an input. We will understand with an example:
Staying in India meant that I gravitated towards cricket. ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access