System Design for the LLM Era

2

Core Architectural Patterns for LLM System Design

In Chapter 1 we introduced the fundamental concepts of LLM integration: tokens, embeddings, and the basic idea of retrieval-augmented generation (RAG). However, integrating LLMs into a production system introduces a new class of dependency: one that is non-deterministic, high-latency, and carries a high, variable operational cost.

As experienced engineers, we already know how to build reliable systems. This chapter isn't about reinventing those principles, but about adapting them for the unique, messy challenges LLMs throw at us. Consider this an architect's playbook that outlines the new patterns needed to meet these challenges. The patterns we'll outline will be the common framework for all ...