Hands-On Generative AI with Transformers and Diffusion Models

Appendix A. Open Source Tools

This book wouldn’t have been possible without open source. Most of the subjects we discussed and the majority of ML research rely on open source contributions—not to mention the production toolchain we used, with open source software such as Jupyter Notebook, Quarto, nbdev, and many more.

In this appendix, we will explore a variety of open source tools for the ML practitioner. Some of these we’ve used in the book, while others are good to know about. By making yourself familiar with these tools, you’ll be well equipped to extend the applications and techniques you just learned.

The Hugging Face Stack

Throughout this book, you’ve become familiar with the core libraries of the Hugging Face stack. These are the two main libraries we used:

transformers

The main library to train and run inference with transformer-based models across modalities. It provides multiple levels of abstraction, from the high-level pipeline and Trainer to supporting running your own PyTorch training loops.

diffusers

Similarly to transformers, the diffusers library allows running pretrained diffusion-based models. Although it’s mostly known for its image-generation capabilities, the library also supports audio, video, and 3D.

Both libraries have an opinionated design that prioritizes usability, simplicity, and customizability. What does this mean for end users? First, both libraries aim to offer consistent specifications across models. Whether you’re using Llama or Gemma ...