Generative AI on Kubernetes

Afterword

The field of generative AI is evolving at an unprecedented pace. Models grow more capable, frameworks emerge and evolve, and best practices shift as the community learns from production deployments. Yet beneath this rapid change lies a stable foundation: Kubernetes has established itself as the platform of choice for operationalizing generative AI workloads at scale. Its ability to orchestrate diverse workload types (from LLM inference services to traditional microservices to data pipelines) makes it uniquely suited for the hybrid applications that define modern generative AI systems.

This is particularly evident in the emerging agentic ecosystem, where generative AI models don’t operate in isolation but instead interact with tools, services, and other models to accomplish complex tasks. These architectures require a platform that can seamlessly integrate generative AI components with business logic, databases, and external APIs. Kubernetes excels at this integration, serving as the connective tissue that brings together all the pieces of an AI-driven application.

Throughout this book, we’ve taken a deliberately practical approach. Rather than following the chronological sequence of activities (where infrastructure decisions about GPU configurations and network topology would come first), we started with what to execute and gradually explored how to run it efficiently. This mirrors how most organizations approach generative AI adoption: you begin by serving existing ...