Mastering Transformers - Second Edition

Table of Contents

Preface

Part 1: Recent Developments in the Field, Installations, and Hello World Applications

1

From Bag-of-Words to the Transformers

Evolution of NLP approaches

Recalling traditional NLP approaches

Language modeling and generation

Leveraging DL

Considering the word order with RNN models

LSTMs and gated recurrent units

Contextual word embeddings and TL

Overview of the Transformer architecture

Attention mechanism

Multi-head attention mechanisms

Using TL with Transformers

Multimodal learning

Summary

References

2

A Hands-On Introduction to the Subject

Technical requirements

Installing transformer with Anaconda

Installation on Linux

Installation on Windows

Installation on macOS

Installing TensorFlow, PyTorch, and Transformer