O'Reilly logo

Deep Learning Cookbook by Douwe Osinga

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Chapter 8. Sequence-to-Sequence Mapping

In this chapter we’ll look at using sequence-to-sequence networks to learn transformations between pieces of text. This is a relatively new technique with tantalizing possibilities. Google claims to have made huge improvements to its Google Translate product using this technique; moreover, it has open sourced a version that can learn language translations purely based on parallel texts.

We won’t go that far to start with. Instead, we’ll start out with a simple model that learns the rules for pluralization in English. After that we’ll extract dialogue from 19th-century novels from Project Gutenberg and train a chatbot on them. For this last project we’ll have to abandon the safety of Keras running in a notebook and will use Google’s open source seq2seq toolkit.

The following notebooks contain the code relevant for this chapter:

08.1 Sequence to sequence mapping
08.2 Import Gutenberg
08.3 Subword tokenizing

8.1 Training a Simple Sequence-to-Sequence Model

Problem

How do you train a model to reverse engineer a transformation?

Solution

Use a sequence-to-sequence mapper.

In Chapter 5 we saw how we can use recurrent networks to “learn” the rules of a sequence. The model learns how to best represent a sequence such that it can predict what the next element will be. Sequence-to-sequence mapping builds on this, but now the model learns to predict a different sequence based on the first one.

We can use this to learn all kinds of transformations. ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required