Prompt Engineering for LLMs

Chapter 4. Designing LLM Applications

The previous two chapters laid the foundations for the remainder of the book. Chapter 2 showed in detail how LLMs function, and we demonstrated that at the end of the day, they are effectively document completion models that predict content one token at a time. Chapter 3 explained how the chat API is built upon the LLMs of Chapter 2. With some syntactic sugar at the API level and a healthy dose of fine-tuning, the document completion model is used to complete conversations between the user and an imagined assistant. When you get down to it, the chat model is really still a document completion model—it’s just that the documents it completes are all conversation transcripts.

From this point forward in the book, you’ll learn everything you need to know about how to build LLM applications to solve problems on behalf of your company and your users. This chapter serves as a gateway to that content. In this chapter, we’ll dive into the LLM application, which you’ll see is actually a transformation layer between the user’s problem domain and the model’s text domain. Furthermore, the LLM application is a transformation layer with a purpose—solving problems!

The Anatomy of the Loop

In Figure 4-1, the LLM application is represented as a loop, meaning an interaction back and forth between the user and the model. The domains of the model and the user are often quite different. The user may be doing any number of things, such as writing an email and looking ...