Evals for AI Engineers

Chapter 12. Improving LLM Agents

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 12th chapter of the final book. If you’d like to be actively involved in reviewing and commenting on this draft, please reach out to the editor at [e-mail].

We have built evaluators, set up CI to catch regressions, and deployed monitoring to track production quality. We have also surfaced failure modes through error analysis (Chapter 3) and data analysis (Chapter 11). Now we use all of that to make the agent better.

In this chapter, you will learn:

• How to improve accuracy through prompt refinement, structural changes, and heavier interventions like fine-tuning

• How to reduce cost through model selection, token reduction, caching, and model cascades

• How to verify that each change actually helps using the evaluators and CI infrastructure from earlier chapters

Both accuracy and cost improvements depend on reliable measurement. Without evaluators (Chapter 5) and monitoring (Chapter 9), we cannot tell whether a change helps or hurts.

Accuracy Optimization

When an agent fails evaluations or frustrates users, accuracy is usually the first thing to fix. We can organize the techniques by effort level: low, medium, and high.

Low Effort: Prompt Refinement

The fastest way to ...