AI and ML for Coders in PyTorch

Chapter 16. Using LLMs with Custom Data

In Chapter 15, we looked at Transformers and how their encoder, decoder, and encoder-decoder architectures work. The results of their revolutionizing NLP can’t be disputed! Then, we looked at transformers, which form the Python library from Hugging Face that’s designed to make it easier to use Transformers.

Large Transformer-based models, which are trained on vast amounts of text, are very powerful, but they aren’t always ideal for specific tasks or domains. In this chapter, we’ll look at how you can use transformers and other APIs to adapt these models to your specific needs.

Fine-tuning allows you to customize pretrained models with your specific data. You could use this approach to create a chatbot, improve classification accuracy, or develop text generation for a more specific domain.

There are several techniques for doing this, including traditional fine-tuning and parameter-efficient tuning with methods like LoRA and parameter-efficient fine-tuning (PEFT). You can also get more out of your LLMs with retrieval-augmented generation (RAG), which we’ll explore in Chapter 18.

In this chapter, we’ll explore some hands-on examples, starting with traditional fine-tuning.

Fine-Tuning an LLM

Let’s take a look, step by step, at how to fine-tune an LLM like BERT. We’ll take the IMDb database and fine-tune the model on it to be better at detecting sentiment in movie reviews. There are a number of steps involved in doing this, so we’ll look at ...