Scaling Search and Retrieval for Contextual AI

Chapter 5. Query Execution and Scoring

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 5th chapter of the final book. Please note that the GitHub repo will be made active later on.

If you’d like to be actively involved in reviewing and commenting on this draft, please reach out to the editor at [e-mail].

By now, we have established a fundamental grasp of how search systems encode and organize data. We’ve seen how raw input, whether textual, numeric, geospatial, or vector, is transformed into indexed representations across immutable segment files. We’ve examined how updates and deletes are implemented using tombstones and versioning, and how the immutable design enables concurrency, consistency, and performance. But indexing is only half the battle. In this chapter, we turn our attention to the other half: the mechanics of querying.

Efficiently executing a query across large-scale, append-only indexes is a non-trivial task. The system must not only find matching documents but do so in a way that is latency-bounded, resource-aware, and accurate enough to support downstream applications such as ranking, recommendations, or retrieval-augmented generation. This requires a sophisticated execution pipeline that includes planning, coordination, filtering, scoring, ...