Advanced Deep Learning with Python

Decoder

In this section, we'll implement the decoder. It follows a pattern that is very similar to the encoder:

class Decoder(torch.nn.Module):    def __init__(self, block: DecoderBlock, N: int, vocab_size: int):        super(Decoder, self).__init__()        self.blocks = clones(block, N)        self.norm = LayerNorm(block.size)        self.projection = torch.nn.Linear(block.size, vocab_size)    def forward(self, x, encoder_states, source_mask, target_mask):        for layer in self.blocks:            x = layer(x, encoder_states, source_mask, target_mask)        x = self.norm(x)        return torch.nn.functional.log_softmax(self.projection(x), dim=-1)

It consists of self.blocks: N instances of DecoderBlock, where the output of each block serves as input to the next. These are followed by the self.norm