Generative AI on Kubernetes

Part IV. AI-Driven Apps

Having covered how to run LLMs for inference and tuning on Kubernetes, we now shift from serving single models to building complete AI-driven applications. LLM services rarely run in isolation: they’re typically integrated within larger systems that orchestrate flows between conversational interfaces, vector databases for context retrieval, and model services for generation.

This part begins with architectural patterns for AI-driven applications, from chat interfaces to event-driven backends, and introduces retrieval-augmented generation (RAG) for grounding model outputs in external knowledge and agentic workflows where LLMs coordinate tool invocation and multistep reasoning. Moving from architecture to operations, we address production challenges unique to agentic systems: security, state management, observability, cost control, and reliability, along with protocols like Model Context Protocol (MCP) and Agent-to-Agent (A2A) that standardize tool and agent communication.

In detail, the chapters in this part cover the following aspects:

• Chapter 8, “AI-Driven Applications”, covers architectural patterns for AI applications including chat, event-driven, and batch workloads, then explores RAG and agentic workflows.

• Chapter 9, “Running Agentic Applications in Production”, addresses production challenges of security, state, observability, cost, and reliability using MCP and A2A protocols.