System Design for the LLM Era

1

Atomic Units of LLM Systems

Software engineering has traditionally been deterministic: if x then y. If a unit test for this passes once, it passes a million times.

Building with large language models (LLMs) requires a fundamental shift to probabilistic engineering. The same input x might yield y, y', or sometimes z. This chapter defines the primitives we use to change this uncertainty into reliable systems.

LLMs are a type of artificial intelligence that have been trained on a vast amount of natural language and programming data which are capable of generating, reasoning and providing solutions. They have gained traction due to their understanding and ability to provide answers to user queries in a wide range of topics, including healthcare, ...