System Design for the LLM Era

Preface

This book is about the system design of AI-powered applications that use large language models (LLMs). It provides full architectural blueprints, from user-facing clients to backend orchestrators and the data layers that support them.

It is not a book on building, training, or researching the internal mechanics of AI/ML models, and neither is it a general system design primer. We assume you understand distributed systems fundamentals (e.g. caching, microservices).

The book's objective is to show you how to integrate and operationalize LLMs within existing scalable software architectures. Thus it is a practical guide to the system design principles, architectural patterns, and operational practices required when integrating LLMs into ...