Transfer Learning for Natural Language Processing

8 Deep transfer learning for NLP with BERT and multilingual BERT

This chapter covers

• Using pretrained Bidirectional Encoder Representations from Transformers (BERT) architecture to perform some interesting tasks
• Using the BERT architecture for cross-lingual transfer learning

In this chapter and the previous chapter, our goal is to cover some representative deep transfer learning modeling architectures for natural language processing (NLP) that rely on a recently popularized neural architecture—the transformer¹—for key functions. This is arguably the most important architecture for NLP today. Specifically, our goal has to look at modeling frameworks such as the generative pretrained transformer (GPT),² Bidirectional Encoder Representations ...