Scalable Kubernetes Infrastructure for AI Platforms

Chapter 2. Model Development on Kubernetes

In this chapter, we will provide an overview of prevailing technologies and techniques for developing machine learning models using Kubernetes as a compute platform. While we will focus on specific techniques relevant to large language models (LLMs) and generative AI, many of the techniques we discuss will apply to traditional predictive models and other architectures as well.

Historically, models have required extensive data preparation to curate high-quality, labeled datasets that sufficiently capture the problem domain. Creating these datasets was very labor-intensive and expensive. More recently, advances in computational power, improved algorithms for distributing training across compute resources, and widespread open access to training data have all paved the way for extremely powerful general-purpose models to be built without heavy data curation.

Generally, foundation LLMs are created via self-supervised learning, a type of unsupervised learning, on extremely large, unlabeled datasets. For LLMs, this results in a model that understands patterns in human language and can predict the most likely output that should follow a given input. These foundational models exhibit usefulness across a wide breadth of tasks, but practitioners often need to adapt these pretrained base models to some specific use case.

There are several prevailing techniques for adapting these foundational models, which differ from each other in their intended ...