Transfer Learning for Natural Language Processing

6 Deep transfer learning for NLP with recurrent neural networks

This chapter covers

• Three representative modeling architectures for transfer learning in NLP relying on RNNs
• Applying these methods to the two problems introduced in the previous chapter
• Transferring knowledge obtained from training on simulated data to real labeled data
• An introduction to some more sophisticated model adaptation strategies via ULMFiT

In the previous chapter, we introduced two example problems for the experiment we will conduct in this chapter—column-type classification and fake news detection. Recall that the goal of the experiment is to study the deep transfer learning methods for NLP that rely on recurrent neural networks (RNNs) for key functions. In particular, ...