Visualizing Generative AI

Chapter 5. Architecting GenAI Applications

Chapter 5

Architecting GenAI Applications

In Chapter 4, we discussed how to build agentic systems, but assumed that each agent involved an LLM call. An architecture that relies on agents that invoke an LLM each and every time results in an underengineered application that will be too expensive and too slow. In this chapter (see Figure 5-1), we look at how to balance engineering complexity, cost, latency, and the risks of using an LLM for a specific use case.

Figure 5-1. Structure of this chapter.

Choosing a Foundational Model

When building a GenAI application, one of the first choices you’ll grapple with is which LLM to use as your foundational model. Figure 5-2 provides a decision tree for this choice.

Figure 5-2. Selecting a foundational LLM, at the time of writing. Use the same process at the time you are reading this book, but the relative scores and model versions might be different.

While it is possible to be LLM-agnostic by using a framework such as LangChain, the reality is not as simple (Figure 5-3). The code will work, but the results will not be the same when you change LLMs. That’s because you will change the prompts and parsing code (even when employing signatures and Pydantic structures, as we recommend in Chapter ...