Hands-On RAG for Production

Chapter 1. Introduction to Retrieval-Augmented Generation (RAG)

Imagine an AI engineer building a simple internal support chatbot for her company. She prototypes it in an afternoon using GPT-5.1, an off-the-shelf large language model (LLM). The early results look impressive: the model converses fluidly, summarizes long documents, and even drafts code. But the moment they ask something grounded in the company’s own systems—“What will be the refund policy after January next year?” or “Which customer accounts are flagged for follow-up this week?”—the answers fall apart. The model confidently returns text that sounds plausible but has no connection to the company’s actual data. Sometimes it fabricates outdated policies. Sometimes it offers complete nonsense. The core issue is not the model’s fluency; it is its blindness.

This is the fundamental limitation of even the most advanced LLMs.¹ They are trained on enormous corpora—books, articles, code repositories, and public web content—which gives them a broad, generalized understanding of language and lets them perform tasks they were never explicitly trained for. But no matter how large the training dataset, it can never contain a company’s private documents, an internal knowledge base, or last week’s updates to a niche library. As a result, LLMs can provide eloquent but incorrect answers whenever a task requires information outside their training window. These errors often manifest as hallucinations: generated content that is fluent ...