Advanced Deep Learning with Python

Implementing seq2seq with attention

In this section, we'll use PyTorch 1.3.1 to implement a simple NMT example with the help of a seq2seq attention model. To clarify, we'll implement a seq2seq attention model, like the one we introduced in the Introducing seq2seq models section, and we'll extend it with Luong attention. The model encoder will take as input a text sequence (sentence) in one language and the decoder will output the corresponding sequence translated into another language.

We'll only show the most relevant parts of the code, but the full example is available at https://github.com/PacktPublishing/Advanced-Deep-Learning-with-Python/tree/master/Chapter08/nmt_rnn_attention. This example is partially based on the PyTorch tutorial ...