Building AI Agent Platforms

Chapter 5. AI Application Runtimes

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 5th chapter of the final book. Please note that the GitHub repo will be made active later on.

If you’d like to be actively involved in reviewing and commenting on this draft, please reach out to the editor at [e-mail].

Civilization advances by extending the number of important operations which we can perform without thinking about them.

Alfred North Whitehead

Once your users have access to models, the barrier to local development vanishes. However, moving from a successful notebook experiment to a resilient production system requires more than just an API key. It requires a runtime.

An AI application runtime provides the execution environment where the “Agentic Quantums” - the bundles of code, prompts, and schemas - live, scale, and interact with the world. While these applications leverage cutting-edge models, their operational heartbeat relies on a familiar pulse: container orchestration.

This chapter explores how to adapt your existing ecosystem of Internal Development Platforms (IDP) and Data Platforms to meet the unique demands of AI, all with the goal of empowering developers to deploy and scale AI applications seamlessly. It starts by discussing the duality of workloads, ...