Hands-On Generative AI with Transformers and Diffusion Models

Chapter 6. Fine-Tuning Language Models

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the sixth chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at [e-mail].

In Chapter 2, we explored how language models work and how to use them for different tasks, such as text generation and sequence classification. We saw that language models could be helpful in many tasks without further training, thanks to proper prompting and the zero-shot capabilities of these models. We also explored some of the hundreds of thousands of pre-trained models by the community. In this chapter, we’ll discuss how we can improve the performance of language models on specific tasks by fine-tuning them on our data.

While pre-trained models showcase remarkable capabilities, their general-purpose training may not be suited for certain tasks or domains. Fine-tuning is frequently used to tailor the model’s understanding to the nuances of their dataset or task. For instance, in the field of medical research, a language model pre-trained on general web text will not perform great out ...