Prompt Engineering for LLMs

Chapter 2. Understanding LLMs

So you want to become the LLM whisperer who unlocks the wealth of their knowledge and processing power with clever prompts? Well, to appreciate which kinds of prompts are clever and tease the right answer from the LLM, you first need to understand how LLMs process information―how they think.

In this chapter, we’ll approach this problem onion style. You’ll first see LLMs from the very outside as trained mimics of text in “What Are LLMs?”. You’ll learn how they split the text into bite-size chunks called tokens in “How LLMs See the World”, and you’ll learn about the fallout if they can’t easily accomplish that split.

You’ll also find out how the token sequences are generated bit by bit in “One Token at a Time”, and you’ll learn about the different ways to choose the next token in “Temperature and Probabilities”. Finally, in “The Transformer Architecture”, you’ll delve into the very inner workings of an LLM, understand it as a collection of minibrains that communicate through a Q&A game called attention, and learn what that means for prompt order.

During all that, please keep in mind that this is a book about using LLMs, not about LLMs themselves. So, there are a lot of cool technical details that we’re not mentioning because they’re not relevant for prompt engineering. If you want matrix multiplications and activation functions, you’ll need to turn elsewhere—the classic reference The Illustrated Transformer is an excellent starting point for a deep ...