Vector Databases for Enterprise AI

Chapter 3. The Vector Pipeline in RAG Systems

While Chapter 2 examined how vector databases behave at query time, this chapter examines what happens after retrieval becomes part of an end-to-end system that must select, filter, and assemble evidence before generation.

To ground this, consider a user asking, “Why did customer churn increase last quarter?” In a retrieval-augmented generation (RAG) system, answering this question typically occurs in two stages. First, the query is converted into an embedding and relevant documents are retrieved from a vector database. Second, the retrieved context is passed to a large language model (LLM), which generates the response using that material as evidence.

During the retrieval stage, the query is embedded, relevant documents are retrieved from the vector database, filters such as permissions and time scope are applied, and a reranking step may reorder the candidates to prioritize the most relevant passages before a bounded context set is assembled. Only after this context is prepared does the LLM generate a response grounded in the retrieved material.

Figure 3-1 illustrates this retrieval and context assembly pipeline, showing how a user query is embedded and how candidate documents are retrieved from the vector database, filtered and reranked, and finally assembled into the bounded context that the LLM uses to generate its response.