Fine-Tuning AI

Chapter 3. The Fine-Tuner’s Workshop

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 3rd chapter of the final book. Please note that the GitHub repo will be made active later on.

If you’d like to be actively involved in reviewing and commenting on this draft, please reach out to the editor at [e-mail].

Have you ever tried to build furniture with the wrong tools? Thankfully the Ikea’s of the world include the necessary allen keys and screwdrivers. But before they started providing those, you usually, at best, ended up with a wobbly desk. And that’s after fighting with your equipment before you actually built anything.

Fine tuning a model is no different. The right hardware and software setup is everything, and the perfect investment to prevent frustrating long nights full of cryptic error messages.

In this chapter, we’ll talk about that workbench. The search space for the correct hardware is near infinite, so I’ll be opinionated about what I think is a good set to begin with! You don’t have to go with my suggestions, of course, but I will be using them through all the examples in this book.

Why Setup Matters

When you run inference on a model, feed a prompt and get a response, you’re doing relatively lightweight work. The model’s weights sit in memory, ...