Building Agentic AI: Workflows, Fine-Tuning, Optimization, and Deployment

9

Optimizing AI Models for Production

Introduction

In Chapter 8, you saw how fine-tuning can ground a model in data and how domain adaptation can bake specialized rules into its parameters. But accuracy alone will not cut it in the wild. Production-level AI demands more than accurate and calibrated answers. It demands speed and cost control and data governance.

In this chapter, we will explore techniques like quantization, which can shrink models in place to use less memory, and distillation, which can teach a smaller model to mimic a larger one. We will also talk about model hosting patterns and how to choose among options such as self-hosting and managed endpoints. From there, we will see how techniques like speculative decoding can create ...