In the previous chapter, you managed to distill all your context into a single, coherent prompt. Now, it’s time for the LLM to do its thing and for you to make sure that it all goes smoothly.
In this chapter, we’re going to start by talking about completion formats and making sure your completions stop when they’re supposed to, as well as how to interpret them using so-called logprob tricks.
Then, we’re going to take a step back so you can ask yourself which model you’re going to choose to invoke: a professional commercial service, an open source alternative, or even your own bespoke fine-tuned model. Time to get into it.
Anatomy of the Ideal Completion
In this section, we’ll examine how completions appear, whether they’re classic completions or chat responses. More importantly, we’ll discuss how you want them to look to ensure clear and effective solutions, all while avoiding issues like unnecessary delays or confusing details. As we did in Chapter 6 with prompts, we’ll break down the components of an LLM completion and go through them one by one (see Figure 7-1).
Figure 7-1. An LLM completion
The Preamble
In the context of completions, the preamble is the initial part of the generated text that sets the stage for the main content. Sometimes, this is helpful, and sometimes, it leads to completions that start with uninteresting or useless detail before ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month, and much more.
