Generative AI on Kubernetes

Introduction

The release of ChatGPT in 2022 was a watershed moment for the IT world. Overnight, it seemed like everything changed, not because of entirely new concepts, but due to the exponential growth in model parameters and the massive expansion of training datasets. Model parameters—the weights and biases learned during training—are often used to measure a model’s complexity and capability. But architectural innovations and training quality are just as important to how well a model actually performs. This combination of scaling parameters and expanding data pushed AI into new territory, with capabilities that were previously unimaginable.

In the world of physics, phase transitions describe moments when small, gradual changes suddenly lead to dramatic shifts in behavior—like water turning to ice. The rise of large language models (LLMs) follows this same pattern. Since the Transformer architecture was introduced in 2017, AI had been steadily evolving, but the leap in model size, compute power, and training data scale pushed it beyond a tipping point. These models began exhibiting human-like text generation and processing, disrupting entire industries and resetting our expectations of what AI can do. The graph in Figure I-1 shows the growth of these parameters and the expanding data sources that have enabled AI’s evolution over the past few years.

Beyond just data, we owe this transformation to advancements in computational power, particularly the widespread adoption of GPUs ...