May 2027
Intermediate to advanced
425 pages
7h 45m
English
Overview
As users increasingly expect AI to interact as naturally as humans, engineers must move beyond static prompts to building systems that perceive, reason, and act instantly, mimicking human interaction more closely. Multimodal, Real-Time AI Agent Systems takes you from agent fundamentals to architecting advanced, bidirectional, multimodal, multi-agent systems, focusing on the difficult leap from proof of concept to production. You'll start by building agents directly with foundation models to understand core components and then master modern agent frameworks that simplify and scale implementation.
Written by industry practitioners, this book connects agentic concepts with cutting-edge standards and protocols of interoperability. You'll learn to build enterprise-grade agent platforms that enforce scalable AgentOps, rigorous evaluation, and extreme security measures required for live streaming interactions.
Through practical multimodal agent examples, you'll understand how to:
- Design scalable, low-latency architectures for single and multi-agent systems
- Engineer the complete lifecycle of live multimodal streaming, using backend-centric architectures to minimize frontend complexity
- Implement unified protocols like the Model Context Protocol and Agent-to-Agent for tool use and agent discovery
- Apply operational best practices for security, testing, and scaling to support thousands of concurrent live agents
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.