Generative AI on Kubernetes

Preface

The end of 2022 marked a turning point in the world of AI with the release of ChatGPT, a chat-based language model designed to generate human-like text in response to conversational input. We all witnessed an AI revolution that transformed our expectations and possibilities. Generative AI models have been around for a while. In fact, deep learning concepts have existed for decades, but it’s only with the recent availability of large amounts of data and advances in accelerators and compute power that this AI revolution finally became possible. This, combined with a massive increase in model parameters reaching billions, has brought about a remarkable shift.

Imagine a phase transition in physics: the same substance suddenly exhibits completely new properties. That’s what happened with AI, revealing new capabilities that were previously unimaginable, such as advanced natural language processing (NLP) and the ability to generate coherent and contextual responses. Small steps in AI development led to significant impacts, as we have seen over the past few years when interest in generative AI models and their diverse applications has exploded. While this early pioneering era is exciting, it is also extremely demanding.

As of early 2026, you can find millions of generative AI models on the Hugging Face Hub, the central repository of the AI community, for various applications. Once you choose a model, the main question for application developers and Machine Learning Operations ...