Inside Deep Learning

13 Transfer learning

This chapter covers

• Transferring a pretrained network to a new problem
• Understanding the difference between frozen and warm weights
• Learning with less data via transfer learning
• Transfer learning for text problems with transformer-based models

You now know a range of techniques for training models from scratch on new data. But what if you do not have time to wait for a big model to train? Or what if you do not have a lot of data to begin with? Ideally, we could use information from a bigger, well-curated dataset to help us learn a more accurate model in fewer epochs for our new, smaller dataset.

That is where transfer learning comes into play. The idea behind transfer learning is that if someone has gone through the effort ...