Large Language Models: The Hard Parts

Chapter 1. First Principles: What to Consider Before We Start Building with LLMs

Freedom is something that dies unless it’s used.

Hunter S. Thompson

Large language models (LLMs) represent a genuine inflection point in what software can do and in how we build applications. LLMs can understand context, recognize patterns across documents, generate coherent explanations, and adapt to nuanced requests without brittle rule sets or exhaustive training data. The capabilities are remarkable, and open source tools have made this technology accessible to organizations of any size and individuals of any budget. Our goal in this book is to arm readers with the open source frameworks to successfully implement this technology and avoid the common pitfalls that can lead us astray.

In the chapters that follow, we walk through discussions, hands-on examples, and reproducible code flows to help us navigate this world. We emphasize not just LLMs but building what we call LLM-based applications, or LLMBAs. It is our belief that the hard parts of LLMs come when we apply them to real-world, tractable problems.

However, before we get to building, let’s spend a few pages on the first principles that come into play before we need to start writing code or architecting frameworks.

In this book we use a variety of tools. Many of the LLMs that we invoke are proprietary and accessed via API. We think it’s valuable to be familiar with invoking those APIs as well as using purely local tools, which we also ...