LLMs today have some major limitations, but that doesn’t mean your dream LLM app is impossible to build. The experience that you design for users of your application needs to work around, and ideally with, the limitations.
Chapter 5 touched on the key trade-off we face when building LLM apps: the trade-off between agency (the LLM’s capacity to act autonomously) and reliability (the degree to which we can trust its outputs). Intuitively, any LLM application will be more useful to us if it takes more actions without our involvement, but if we let agency go too far, the application will inevitably do things we wish it hadn’t.
To borrow a concept from other fields,1 we can visualize the trade-off as a frontier—all points on the frontier’s curved line are optimal LLM architectures for some application, marking different choices between agency and reliability. (Refer to Chapter 5 for an overview of different LLM application architectures.) As an example, notice how the chain architecture has relatively low agency but higher reliability, whereas the Agent architecture has higher agency at the expense of lower reliability.
Let’s briefly touch on a number of additional (but still important) objectives that you might want your LLM application to have. Each LLM ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month, and much more.
