book

RAG from First Principles

Name: RAG from First Principles
Author: Jia Huang
ISBN: 9781835888667

by Jia Huang

May 2026

Intermediate

492 pages

12h 5m

English

Packt Publishing

Read now

Unlock full access

Contributors
Subscribe for a free ebook
Preface
Who this book is forWhat this book coversGet in touchFree benefits with your bookHow to unlockJoin our Discord and Reddit Space
Data Import
Does the parsing process depend on file type?Reading simple text with a DataLoaderUsing LangChain to read TXT files and generate document objectsData loaders in LangChainUsing LangChain to read all files in a directoryUsing LlamaIndex to read all documents in a directoryConnect learner with LlamaHub and read database entriesReading various types of documents with the unstructured toolParsing specific elements with the JSON loaderReading text from imagesReading text using UnstructuredLoaderReading text using UnstructuredImageLoaderReading text from a PPTUsing large models for holistic image-text parsingWukong titleBodyImage contentPrologue titleBodyImage contentAdaptations from Journey to the WestBodyImage contentImporting table data in CSV formatImporting data using CSVLoaderComparing CSVLoader and UnstructuredCSVLoaderCrawling and parsing web documentsParse web pages with WebBaseLoaderGranular webpage parsing with UnstructuredLoaderUnderstanding the cultural and story background of Black Myth: WukongMarkdown file titles and structureWhy Markdown?Implementing UnstructuredMarkdownLoaderText formats, layout recognition, and table parsing in PDF filesUsing loading tools for PDF filesSimple text extraction with PyPDFLoaderUsing the Marker tool to convert PDF documents to Markdown formatStructured parsing with UnstructuredLoaderVisualizing layout with PyMuPDF and coordinate informationUsing UnstructuredLoader to parse tables in PDF pagesIntegrating content under the same title using ParentIDSummaryGet this book’s PDF version and more
Text Chunking
Why chunking is very importantContext window limits the maximum chunk lengthImpact of chunk size on retrieval accuracyImpact of chunk size on generation qualityHow small can the chunk size be while still retaining enough context?Different chunking strategiesVisualize chunking with the ChunkViz toolChunking by a fixed number of charactersThe CharacterTextSplitter tool in LangChainSetting chunk size parameters in LlamaIndexRecursive chunkingChunking based on specific formatsChunking based on document structure or semanticsUsing the unstructured tool for document structure-based chunkingUsing LlamaIndex SemanticSplitterNodeParser for semantic chunkingAdvanced indexing techniques related to chunkingSentence splitting with sliding windowMixed generation of parent and child text blocks during chunkingCreating metadata for text chunks during chunkingWorkflow of HyDE technologyForming hierarchical indices during chunkingDocument embedded object technology for complex data structuresSummarySubscribe for a free eBook
Information Embedding
Embedding: Encoding external informationMeasuring similarity between vectorsFrom early word embedding models to large model embeddingsEarly word embedding modelsContextual word embedding modelsSentence embedding models and the SentenceTransformers frameworkComputing semantic similarity between two sentencesMultilingual embedding modelsEmbedding models for different data typesImage, audio, and video embedding modelsJoint image and text embedding modelsGraph and knowledge graph embedding modelsEmbedding models in the era of large language modelsModern embedding models: OpenAI, Jina, Cohere, VoyageUsing OpenAI’s text-embedding-3-small for product recommendationsExample product recommendation scenario using embeddingsUsing Jina embeddings for cross-lingual clusteringEvaluating embedding models with MTEBComparing and selecting embedding modelsSparse embedding, dense embedding, and BM25Understanding sparse and dense embeddingsImplementing sparse embeddings with BM25Hybrid retrieval with BGE-M3How BGE-M3 generates sparse and dense embeddingsMultimodal embedding model visualized_BGEUsing embedding models with frameworks like LangChain and LlamaIndexEmbedding interfaces in LangChainEmbedding interfaces in LlamaIndexEmbedding caching via LangChain cacheFine-tuning embedding modelsSummaryGet this book’s PDF version and more
Vector Storage
How are vectors storedA simple vector index from the perspective of LlamaIndex’s designComponents of a vector databaseIndexing in vector databasesFlat indexIVF indexQuantization indexGraph indexingHashing techniquesVector retrieval (similarity measurement)Mainstream vector databasesSelection and evaluation of vector databasesVector database selectionIndex and search settings in vector databasesMilvus vector operation examplePreparing the sample datasetCreating a Milvus collection and indexExample: Inserting and querying monster data with vector searchChoosing the appropriate index typeFLAT (Exact brute-force search)IVF_FLAT (Inverted file index + Exact search)IVF_PQ (Inverted file index + Product quantization)ScaNNHNSWHNSW_SQ, HNSW_PQ, and HNSW_PRQSPARSE_INVERTED_INDEXBIN_FLAT and BIN_IVF_FLATDisk index and GPU indexSelecting the appropriate metricCommonly used metrics for dense floating-point vectorsThe metric used in search must match the indexSearch and query: Two retrieval methodsUsing Milvus for hybrid searchFloating point vectors, sparse floating point vectors, and binary vectorsHybrid retrieval strategy implementationBuilding a hybrid retrieval system using MilvusVector databases and multimodal retrievalUsing the visualized BGE model for multimodal retrievalLoading the visualized BGE modelCreating the image datasetGenerating image embeddingsStoring image vectors in MilvusCreating a multimodal search functionExecuting a sample image search queryVisualizing the search resultsUsing ResNet-34 to extract image features and perform retrievalExtracting image features with ResNet-34Storing scene features in MilvusSearching for similar scenesData maintenance and vector storage CRUD operations in RAG systemsData flow maintenance and management in RAG systemsVector insert, delete, and update operations in MilvusInserting vector entitiesUpserting vector entitiesDeleting vector entitiesCollection operations in vector databasesViewing collection informationLoad collection into memoryRelease memory resourcesDeleting a collectionSummary
Pre-Retrieval Processing
Why pre-retrieval processing mattersTechniques included in pre-retrieval processingQuery construction: Asking questions in natural languageText-to-SQL: Transforming natural language into SQLWhy Text-to-SQL matters in RAGTraditional Text-to-SQL implementationText-to-SQL in the deep learning eraLarge language model-based Text-to-SQL implementationExample: Building a simple Text-to-SQL workflowSolution combining agent, SQL data function, and tool callsHow to understand metadata extractionWhat are data functionsTwo-turn conversational query processText-to-cypher: From natural language to graph database queriesUnderstanding graph databasesCore components of a graph databaseWhy graph databases matter in RAGBasic Cypher syntaxText-to-Cypher workflowExample: Generating a Cypher query from natural languageSQL query vs Cypher querySelf-query retriever automatically generating metadata filter conditions from queriesMetadata fields for video retrievalCombining vector retrieval and metadata filteringInstalling required librariesLoading documents with metadataBuilding the self-query retrieverConfigure metadata fields for the retrieverLimitations of Self-query retrieverQuery translation: Better explaining user questionsQuery rewriting: Reformulating the original question into an appropriate formGuiding large language models to rewrite queries via promptsOptimizing queries with RePhraseQueryRetrieverQuery decomposition: Breaking down queries into multiple subproblemsUsing MultiQueryRetrieverQuery clarification: Step-by-step refinement and clarification of user questionsQuery expansion using HyDE to generate hypothetical documentsQuery routing finding the right data sourceLogical routing determining the path of the queryExample: Routing to different data sourcesSemantic routing: Selecting relevant promptsExample: Routing to prompt templatesSummaryGet this book’s PDF version and more
Index Optimization
From small to large node-sentence sliding window and parent-child text chunksNode-sentence sliding window retrievalParent-child text chunk retrievalBuilding summary-to-detail indexes with IndexNode and RecursiveRetrieverImplementation of the Summary-to-Detail strategy in LlamaIndexHierarchical merging HierarchicalNodeParser and RAPTORCreating hierarchical indexes with HierarchicalNodeParserUsing RAPTOR to recursively generate multi-level indexesForward/backward linking connecting related nodes by forward/backward extensionHybrid retrieval improving retrieval accuracy and expanding coverageCombining BM25 and semantic retrieval with ensemble retrieverUsing MultiVectorRetriever to implement multi-representation indexingHybrid querying and query routingSummarySubscribe for a free eBook
Retrieval Post-Processing
Common retrieval post-processing techniquesRerankingRRF re-rankingHow should I understand the smoothing parameter?Cross-encoder re-rankingColBERT re-rankingCohere re-ranking and Jina re-rankingIntroduction to Jina reranker v2Applications and capabilitiesPerformance and efficiencyIntegration and accessRankGPT and RankLLMTime-weighted re-rankingDifference between LangChain and LlamaIndex recency weightingCompressionContextual compression retrieversCompressing prompts with LLMLinguaHow to use LLMLingua to compress promptsRECOMP methodSentence embedding optimizerRemembering long context through prompt cachingCorrectionSummaryGet this book’s PDF version and more
Response Generation
Improving LLM output quality by refining promptsGuiding generation results with templates and examplesEnhancing the diversity and comprehensiveness of generationIntroducing fact-checking mechanisms to enhance factual accuracyControlling output format through output parsingLangChain output parsing mechanismsLlamaIndex output parsing mechanismsBasic parsing modeStructured parsing modeOpenAI’s JSON mode and structured outputPydantic parsingFunction call parsingImproving output quality through model choiceMethods of integrating retrieved results into the generation processInput layer integrationOutput layer integrationIntermediate layer integrationSelf-RAG generationRRR dynamic generation optimizationSummarySubscribe for a free eBook

System Evaluation
Evaluation system for RAG systemsEvaluation dataset for RAGRetrieval evaluation and response evaluationRAG TRIAD – holistic evaluationRetrieval evaluation metricsPrecisionRecallF1 scoreMean reciprocal rankMean average precisionP@KDocument precision, page precision, and positional document precisionPositional document precisionResponse evaluation metricsN-gram overlap-based metricsMetrics based on semantic similarityMetrics based on faithfulness or groundednessEvaluation frameworks for RAG systemsUsing RAGAS to evaluate RAG systemsUsing TruLens for RAG TRIAD evaluationDeepEval – a powerful open-source large model evaluation frameworkPhoenix – interactive model diagnostics and analysis platformSummaryGet this book’s PDF version and more
Complex RAG Paradigms
An overview of complex RAG paradigmsGraphRAG integration of RAG and knowledge graphsContextual retrieval: breaking through the contextual challenges of traditional RAGModular RAG: transitioning from fixed processes to flexible architecturesAgentic RAG or agent-driven RAG systemsMultimodal RAGSummarySubscribe for a free eBook
Unlock Your Exclusive Benefits
Unlock this Book’s Free Benefits in 3 Easy StepsStep 1Step 2Step 3Need Help
Other Books You May Enjoy
Index

Content preview from RAG from First Principles

2 Text Chunking

After importing unstructured data, we need to perform text chunking, also known as text splitting. This process involves dividing long texts into appropriately sized fragments to facilitate embedding, indexing, and storage, and to improve retrieval accuracy.

Figure 2.1: Text chunking optimization methods with flowcharts

Lewis: Why do we need to do this step? It’s not hard to understand, right?

Alex: Exactly. The results of retrieval are composed of individual units, which we call chunks. The size of these units is very important. Take Journey to the West as an example; this work is nearly a million words long. If a user asks what ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781835888667

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

RAG from First Principles

by Jia Huang

2

Text Chunking

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.