LLMOps

Preface

I’ve lost count of how many times I’ve been asked, “What’s the difference between an LLM/AI engineer and an LLMOps engineer?” It’s one of those questions that keep popping up, whether I’m in a meeting, at a conference, or just grabbing coffee with someone in the field.

I used to start by explaining the technical distinctions between the roles. But over time, I realized the real issue: people don’t fully grasp what it takes to keep the lights on with large language models (LLMs) in production over an extended period.

As I write this in early 2025, the top models, techniques, and best practices are changing every few days. Thus, very few people understand their complexity. Most people still think of operationalizing, or “Ops,” as deployment, but in the LLM context, Ops is really about streamlining people, processes, and technology to make these models secure, robust, and reliable in production.

Enterprises and their human resource departments are scrambling to figure out what it all means for their teams and their projects, and in this book I have done my best to answer that question. This book isn’t a tutorial on defining roles or how to build and deploy an LLM; while it touches on both of those topics, that isn’t enough anymore. Once LLM-based applications are in production, someone has to keep them optimized, or they risk becoming overengineered solutions to simple problems or, worse, badly maintained houses of cards that crumble under high demand or a prompt injection ...