Machine Learning with PyTorch and Scikit-Learn

16

Transformers – Improving Natural Language Processing with Attention Mechanisms

In the previous chapter, we learned about recurrent neural networks (RNNs) and their applications in natural language processing (NLP) through a sentiment analysis project. However, a new architecture has recently emerged that has been shown to outperform the RNN-based sequence-to-sequence (seq2seq) models in several NLP tasks. This is the so-called transformer architecture.

Transformers have revolutionized natural language processing and have been at the forefront of many impressive applications ranging from automated language translation (https://ai.googleblog.com/2020/06/recent-advances-in-google-translate.html) and modeling fundamental properties of protein ...