August 2026
Intermediate
368 pages
11h 8m
English
Overview
“Practical, comprehensive, and urgently needed.”
—Samer Hamad, Amazon
Building Reliable AI Systems: Applications and agents you can trust shows you how to grow a promising prototype into a production product you can deliver, maintain, and scale. In 11 sharply-focused chapters, it walks you through a unique reliability process that author Rush Shahani refined while building an AI-powered sales intelligence platform used by over 15,000 go-to-market teams. Timely and practical, this book provides concrete steps to eliminate costly hallucinations, streamline computational resources, and confidently deploy secure, trustworthy, and compliant AI solutions.
You’ll immediately appreciate how Building Reliable AI Systems defines “AI reliability” along six critical dimensions—accuracy and grounding, safe agency, graceful failure, consistency, fairness, and operational efficiency—and provides a specific three-layer framework to achieve these goals. The first layer, with a focus on outputs, shows you how to get consistent and accurate responses using well-engineered prompts, RAG, and model customization. The second layer, about agents, establishes parameters for memory, tool usage, and orchestration in agentic systems. The book concludes with the third layer—Reliable Operations—covering deployment, monitoring, and responsible AI.
The book’s reliability framework emphasizes the symbiotic relationship between evaluation and reliability, proving that you cannot improve what you do not measure. You’ll learn to measure your applications using fine-grained LLM-native evaluation rubrics like the Grounding Defect Rate, Hallucination Severity Score, and FActScore that audit everything from RAG outputs to the entire step-by-step reasoning trajectory of autonomous agents.
Building Reliable AI Systems stays rooted in reality from start to finish. As you go, you will build several projects, including a multi-agent travel planner and a medical assistant, and use industry standard tools and specifications like LangGraph and MCP. In the helpful appendices you’ll find a handy reference architecture and decision-making checklists.
Reviewer Mohamed Zohir Koufi, Lead AI Engineer at Capgemini, praises how the book “brings together architecture, tooling, evaluation, and governance in the way real systems are built.” By taking this comprehensive approach to LLMOps, the book delivers a reproducible process to ensure your applications stay cost-effective, fast, and compliant with enterprise standards like HIPAA and GDPR.
About the Technology
About the Book
What's Inside
About the Reader
For Python-fluent software engineers and data scientists ready to build production-grade, reliable AI applications.
About the Author
Rush Shahani is an AI leader who has spent his career shipping machine learning systems that hold up under real-world conditions. He co-founded Persana AI, a Y Combinator-backed sales intelligence platform that has joined forces with Rox. Before Persana, Rush built AI and search systems at LinkedIn and Element AI (acquired by ServiceNow), and backend and payments infrastructure at Shopify.
Quotes
A resource packed with actionable advice.
- Attila Bagossy, Sustainabot
Must-read. Offers clear explanations, practical techniques, and real-world insights bridging the gap between theory and application.
- Nehaa Bansal, SAP Labs
An indispensable guide for the next era of AI engineering.
- Oren Etzioni, University of Washington
A thoroughly researched book that will spark the interest of many readers.
- Georg Sommer, HTL Spengergasse
The dual perspective—combining implementation-level details with real-world product considerations—makes the writing particularly engaging and relevant for practitioners building production-grade LLM systems.
- Heng Zhang, Salesforce
Helps you stay up to date and valid on the hottest topic in the engineering community right now.
- Clyde Kallahan, Unlocked Labs
—Samer Hamad, Amazon
Building Reliable AI Systems: Applications and agents you can trust shows you how to grow a promising prototype into a production product you can deliver, maintain, and scale. In 11 sharply-focused chapters, it walks you through a unique reliability process that author Rush Shahani refined while building an AI-powered sales intelligence platform used by over 15,000 go-to-market teams. Timely and practical, this book provides concrete steps to eliminate costly hallucinations, streamline computational resources, and confidently deploy secure, trustworthy, and compliant AI solutions.
You’ll immediately appreciate how Building Reliable AI Systems defines “AI reliability” along six critical dimensions—accuracy and grounding, safe agency, graceful failure, consistency, fairness, and operational efficiency—and provides a specific three-layer framework to achieve these goals. The first layer, with a focus on outputs, shows you how to get consistent and accurate responses using well-engineered prompts, RAG, and model customization. The second layer, about agents, establishes parameters for memory, tool usage, and orchestration in agentic systems. The book concludes with the third layer—Reliable Operations—covering deployment, monitoring, and responsible AI.
The book’s reliability framework emphasizes the symbiotic relationship between evaluation and reliability, proving that you cannot improve what you do not measure. You’ll learn to measure your applications using fine-grained LLM-native evaluation rubrics like the Grounding Defect Rate, Hallucination Severity Score, and FActScore that audit everything from RAG outputs to the entire step-by-step reasoning trajectory of autonomous agents.
Building Reliable AI Systems stays rooted in reality from start to finish. As you go, you will build several projects, including a multi-agent travel planner and a medical assistant, and use industry standard tools and specifications like LangGraph and MCP. In the helpful appendices you’ll find a handy reference architecture and decision-making checklists.
Reviewer Mohamed Zohir Koufi, Lead AI Engineer at Capgemini, praises how the book “brings together architecture, tooling, evaluation, and governance in the way real systems are built.” By taking this comprehensive approach to LLMOps, the book delivers a reproducible process to ensure your applications stay cost-effective, fast, and compliant with enterprise standards like HIPAA and GDPR.
About the Technology
About the Book
What's Inside
- Ground outputs in real business data
- Orchestrate safe, consistent multi-step agent workflows
- Implement robust monitoring, semantic caching, and multi-model fallbacks
About the Reader
For Python-fluent software engineers and data scientists ready to build production-grade, reliable AI applications.
About the Author
Rush Shahani is an AI leader who has spent his career shipping machine learning systems that hold up under real-world conditions. He co-founded Persana AI, a Y Combinator-backed sales intelligence platform that has joined forces with Rox. Before Persana, Rush built AI and search systems at LinkedIn and Element AI (acquired by ServiceNow), and backend and payments infrastructure at Shopify.
Quotes
A resource packed with actionable advice.
- Attila Bagossy, Sustainabot
Must-read. Offers clear explanations, practical techniques, and real-world insights bridging the gap between theory and application.
- Nehaa Bansal, SAP Labs
An indispensable guide for the next era of AI engineering.
- Oren Etzioni, University of Washington
A thoroughly researched book that will spark the interest of many readers.
- Georg Sommer, HTL Spengergasse
The dual perspective—combining implementation-level details with real-world product considerations—makes the writing particularly engaging and relevant for practitioners building production-grade LLM systems.
- Heng Zhang, Salesforce
Helps you stay up to date and valid on the hottest topic in the engineering community right now.
- Clyde Kallahan, Unlocked Labs
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.