O'Reilly logo

Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition by Aurélien Géron

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Chapter 16. Natural Language Processing with RNNs and Attention

When Alan Turing imagined his famous Turing test1 in 1950, his objective was to evaluate a machine’s ability to match human intelligence. He could have tested for many things, such as the ability to recognize cats on pictures, play chess, compose music or escape a maze, but, interestingly, he chose a linguistic task. More specifically, he devised a chatbot capable of fooling its interlocutor into thinking it is human.2 For sure, this test has its weaknesses: a set of hard-coded rules can fool unsuspecting or naive humans (e.g., the machine could give vague pre-defined answers in response to some keywords, it could pretend that it is joking or drunk, to get a pass on its weirdest answers, or it could escape difficult questions by answering them with its own questions), and many aspects of human intelligence are utterly ignored (e.g., the ability to interpret non verbal communication such as facial expressions, or to learn a manual task). But the test does highlight the fact that mastering language is arguably Homo sapiens’s greatest cognitive ability. Can we build a machine that can read and write natural language?

A common approach for natural language tasks is to use Recurrent Neural Networks. We will therefore continue to explore RNNs (introduced in Chapter 15), starting with a Character-RNN, trained to predict the next character in a sentence. This will allow us to generate some original text, and in the process ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required