Advanced Deep Learning with Python

The transformer model

Now that we are familiar with multihead attention, let's focus on the full transformer model, starting with the following diagram:

The transformer model architecture. The left-hand side shows the encoder and the right-hand side shows the decoder; source: https://arxiv.org/abs/1706.03762

It looks scary, but fret not—it's easier than it seems. We'll start with the encoder (the left-hand component of the preceding diagram):

• It begins with an input sequence of one-hot-encoded words, which are transformed into d_model-dimensional embedding vectors. The embedding vectors are further multiplied by .
• The transformer doesn't use ...