book

Unlocking Data with Generative AI and RAG - Second Edition

Name: Unlocking Data with Generative AI and RAG - Second Edition
Author: Keith Bourne
ISBN: 9781806381654

by Keith Bourne

December 2025

Intermediate to advanced

606 pages

15h 23m

English

Packt Publishing

Read now

Unlock full access

Contributors
Subscribe for a free eBook
Preface
Who this book is forWhat this book coversTo get the most out of this bookDisclaimer on imagesGet in touchJoin our Discord and Reddit spaceFree Benefits with Your BookHow to Unlock
Part 1: Introduction to Retrieval-Augmented Generation (RAG)
What Is Retrieval-Augmented Generation?
Free benefits with your bookUnderstanding RAG – basics and principlesAdvantages of RAGChallenges of RAGRAG vocabularyLLMPrompting, prompt design, and prompt engineeringLangChain and LlamaIndexInferenceAI agents and agent-related terminologyContext windowFine-tuning – full-model fine-tuning and parameter-efficient fine-tuningVector store or vector database?Vectors, vectors, vectors!Understanding vectorsImplementing RAG in AI applicationsComparing RAG with conventional generative AIComparing RAG with model fine-tuningThe architecture and stages of RAG systemsSummaryGet This Book’s PDF Version and Exclusive Extras
Code Lab: An Entire RAG Pipeline
Technical requirementsSetting up an LLM account with OpenAICode lab 2.1 – building your first RAG pipelineStep 1 – installing dependenciesStep 2 – importsStep 3 – OpenAI connectionStep 4 – indexingWeb loading and crawlingStep 5 – splittingStep 6 – embedding and indexing the chunksRetrieval and generationStep 7 – prompt templates from LangChain HubStep 8 – formatting a function so that it matches the next step’s inputStep 9 – defining your LLMStep 10 – setting up a LangChain chain using LCELStep 11 – submitting a question for RAGFinal outputSummarySubscribe for a free eBook
Practical Applications of RAG
Customer support and chatbots with RAGTechnical supportFinancial servicesHealthcareRAG for automated reportingHow RAG is utilized with automated reportingTransforming unstructured data into actionable insightsEnhancing decision-making and strategic planningE-commerce supportDynamic online product descriptionsProduct recommendations for e-commerce sitesUtilizing knowledge bases with RAGSearchability and utility of internal knowledge basesExternal knowledge retrieval: from compliance to competitive intelligencePersonalization and targeted recommendationsTraining and educationCode lab 3.1 – adding sources to your RAGSummaryGet This Book’s PDF Version and Exclusive Extras
Components of a RAG System
Technical requirementsKey component overviewIndexingRetrieval and generationRetrieval-focused stepsThe generation stagePromptingDefining your LLMUser interface or UIPre-processingPost-processingOutput interfaceEvaluationSummaryReferenceSubscribe for a free eBook
Managing Security in RAG Applications
Technical requirementsHow RAG can be leveraged as a security solutionLimiting dataEnsuring the reliability of generated contentMaintaining transparencyRAG security challengesLLMs as black boxesPrivacy concerns and protecting user dataHallucinationsRed teamingCommon areas to target with red teamingResources for building your red team planCode lab 5.1 – securing your keysWhy hide your keys?The env.txt approachProtecting your versioning systemStoring multiple secretsA note on kernel restartsCode lab 5.2 – red team attack!Code lab 5.3 – blue team defend!SummaryGet This Book’s PDF Version and Exclusive Extras
Part 2: Components of RAG
Interfacing with RAG and Gradio
Technical requirementsWhy Gradio?Benefits of using GradioLimitations to using GradioCode lab – adding a Gradio interfaceSummarySubscribe for a free eBook

The Key Role Vectors and Vector Stores Play in RAG
Technical requirementsFundamentals of vectors in RAGWhat is the difference between embeddings and vectors?What is a vector?Vector dimensions and sizeWhere vectors lurk in your codeVectorization occurs in two placesVector databases/stores store and contain vectorsVector similarity compares your vectorsThe amount of text you vectorize matters!Not all semantics are created equal!Code lab 7.1 – Common vectorization techniquesTerm frequency-inverse document frequency (TF-IDF)Word2Vec, Sentence2Vec, and Doc2VecBidirectional encoder representations from transformersOpenAI and other similar large-scale embedding servicesFactors in selecting a vectorization optionQuality of the embeddingCostNetwork availabilitySpeedEmbedding compatibilityGetting started with vector storesData sources (other than vector)Vector storesA note on terminology – vector stores versus vector databasesVector store architectureCommon vector store optionsChromaLanceDBMilvuspgvectorPineconeWeaviateChoosing a vector storeSummaryGet This Book’s PDF Version and Exclusive Extras
Similarity Searching with Vectors
Technical requirementsDistance metrics versus similarity algorithms versus vector searchVector spaceSemantic versus keyword searchSemantic search exampleCode lab 8.1 – semantic distance metricsEuclidean distance (L2)Dot product (also called inner product)Cosine distanceDifferent search paradigms – sparse, dense, and hybridDense searchSparse searchHybrid searchCode lab 8.2 – Hybrid search with a custom functionCode lab 8.3 – Hybrid search with LangChain’s EnsembleRetriever retriever to replace our custom functionSemantic search algorithmsk-NNANNEnhancing search with indexing techniquesVector search optionspgvectorElasticsearchFAISSGoogle Vertex AI Vector SearchAzure AI SearchApproximate Nearest Neighbors Oh YeahPineconeWeaviateChromaSummarySubscribe for a free eBook
Evaluating RAG Quantitatively and with Visualizations
Technical requirementsEvaluating as you buildEvaluating after you deployEvaluation helps you get betterStandardized evaluation frameworksEmbedding model benchmarksVector store and vector search benchmarksLLM benchmarksFinal thoughts on standardized evaluation frameworksWhat is the ground truth?How do you use the ground truth?Generating the ground truthHuman annotationExpert knowledgeCrowdsourcingSynthetic ground truthCode lab 9.1 – ragasSetting up LLMs/embedding modelsGenerating the synthetic ground truthAnalyzing the ragas resultsRetrieval evaluationGeneration evaluationEnd-to-end evaluationOther component-wise evaluationRagas founder insightsAdditional evaluation techniquesBilingual Evaluation Understudy (BLEU)Recall-Oriented Understudy for Gisting Evaluation (ROUGE)Semantic similarityHuman evaluationSummaryReferencesGet This Book’s PDF Version and Exclusive Extras
Key RAG Components in LangChain
Technical requirementsCode lab 10.1 – LangChain vector storeVector stores, LangChain, and RAGChromaFAISSWeaviateCode lab 10.2 – LangChain retrieversRetrievers, LangChain, and RAGBasic retriever (dense embeddings)Similarity score threshold retrievalMMRBM25 retrieverEnsemble retrieverWikipedia retrieverkNN retrieverCode lab 10.3 – LangChain LLMsLLMs, LangChain, and RAGOpenAITogether AIExtending the LLM capabilitiesAsyncStreamBatchSummarySubscribe for a free eBook
Using LangChain to Get More from RAG
Technical requirementsCode lab 11.1 – document loadersCode lab 11.2 – text splittersCharacter text splitterRecursive character text splitterCode lab 11.3 – output parsersString output parserJSON output parserSummaryGet This Book’s PDF Version and Exclusive Extras
Part 3: Implementing Agentic RAG
Combining RAG with the Power of AI Agents and LangGraph
Technical requirementsFundamentals of AI agents and RAG integrationLiving in an AI agent worldLLMs as the agents’ brainsGraphs, AI agents, and LangGraphCode lab 12.1 – adding a LangGraph retrieval agent to RAGTools and toolkitsAgent stateCore concepts of graph theoryNodes and edges in our agentCyclical graph setupSummarySubscribe for a free eBook
Ontology-Based Knowledge Engineering for Graphs
Technical requirementsIntroduction to ontologies and knowledge engineeringWhat is an ontology?The role of ontologies in agent architecturesEncoding domain expertiseOntology modeling languages: RDFS versus OWLCode Lab 13.1 – building a simple financial ontology in ProtégéWhat is Protégé?Step 1 – define domain, scope, purpose, and competency questionsStep 2 – gather and analyze domain knowledgeStep 3 – establish a preliminary class hierarchyStep 4 – define properties and relationshipsStep 5 – choose an ontology modeling languageStep 6 – install Protégé 5.6.5Step 7 – Launch Protégé, rename the ontology IRI, and save it using the correct syntaxStep 8 – build the class hierarchy in Protégé using the bulk entry wizardBasic Protégé terminologyPopulating all classes in one goStep 9 – manually adding one class at a timeStep 10 – define propertiesStep 11 – create reference individuals firstStep 12 – create RegulatoryAuthority individual(s)Step 13 – create stock individuals and linking propertiesStep 14 – enrichment and annotationWhy this lab mattersSummary
Graph-Based RAG
Introduction to graph-based RAGCyclical graphs versus ontology-based KGsCode lab 14-1 – building and querying a financial ontology KG in Neo4j (from Protégé)Designing and implementing ontology-driven KGs in Neo4jCore graph concepts refresherStep 1 – a recap: what you have in ProtégéStep 2 – preparing your notebook environmentStep 3 – converting your Protégé ontology for Neo4j importStep 4 – importing nodes, edges, and data properties into Neo4jStep 5 – add navigational anchor nodes (stocks, bonds, and so on)Step 5.2 – create All X concept nodes and wire membersStep 6 – enable hybrid embeddings (text + structure) and multi-hop supportStep 7 – vector search and graph expansion (ready-to-prompt context)Step 8 – generate with LangChain and OpenAIBest practices and next stepsSummarySubscribe for a free eBook
Semantic Caches
Technical requirementsUnderstanding semantic caches with the long-tail patternSemantic cache – the interceptor layerCore components of semantic cachesThe intelligence layer between the query and the responseWhy use a semantic cache?Proper semantic cache planningOrchestrator/planner systemsRule-based fallback systemsIntent classification pathwaysRetrieval, population, and eviction policiesCode lab 15.1 – using semantic cache for retrievalStep 1 – installing dependenciesStep 2 – setting upStep 3 – basic semantic cacheStep 4 – entity masking for better generalizationStep 5 – cross-encoder verificationStep 6 – adaptive thresholdsStep 7 – auto-population with fallbackPutting it all togetherPopulation – query expansion strategiesLLM-based paraphrasingBack-translation for natural variationSynonym and lexical expansionSynthetic query generationDomain-specific constraintsEviction – keeping a clean cacheTime-based evictionLeast recently used with semantic decayPerformance-based pruningSemantic clustering for redundancy eliminationSummaryGet This Book’s PDF Version and Exclusive Extras
Agentic Memory: Extending RAG with Stateful Intelligence
What is agentic memory?The evolution of agent memoryThe pre-ChatGPT era – state machines and slot fillingThe ChatGPT inflection point – from stateless to statefulFrom simple dichotomies to cognitive architecturesCore memory types (CoALA framework)Working memory (short-term)Episodic memory (experiences/events)Semantic memory (facts/knowledge)Procedural memory (skills)Memory scope dimensionsCommunity/public memory – shared across all usersPersonal/user-specific memory – individual contextHow memory types and scopes intersectChallenges of implementing CoALAQuality of working memory dataCollecting comprehensive user experience eventsCuration and maintenance of growing memory storesComparing three memory framework approachesMem0 architectureLangMemZep and GraphitiChoosing the right memory frameworkEvaluation and monitoringEvaluating memory-based versus memoryless systemsMemory effectiveness metricsRetrieval metricsStorage metricsBehavioral evaluationConversation coherence metricsLearning and adaptation metricsTemporal consistency metricsMulti-hop and cross-session reasoningPerformance trackingReal-time monitoring requirementsLongitudinal analysis patternsAutomated maintenance strategiesSummarySubscribe for a free eBook
RAG-Based Agentic Memory in Code
Technical requirementsCode lab 17.1 – setting up the initial agentStep 1 – installing the required packagesStep 2 – importing libraries and configuring the environmentStep 3 – building a basic agentStep 4 – testing the basic agentCode lab 17.2 – coding episodic memory componentsStep 1 – importing additional dependencies for episodic memoryStep 2 – creating episodic memory storage functionsStep 3 – enhancing the agent node with episodic recallStep 4 – building a memory-enabled workflowStep 5 – testing our episodic memory functionalityReal-world application – customer service botCode lab 17.3 – coding semantic memory componentsStep 1 – importing dependencies for semantic memoryStep 2 – defining a semantic fact structureStep 3 – building semantic memory retrievalStep 4 – creating a semantic memory-aware agentStep 5 – testing semantic memory functionalityReal-world application – educational tutorReal-world application – personal assistantSummaryGet This Book’s PDF Version and Exclusive Extras
Procedural Memory for RAG with LangMem
Technical requirements Benefits of using procedural memoryCode lab 18.1 – implementing procedural memory optimizationStep 1 – setting up the foundation with imports and a baseline agentStep 2 – define procedure structure for hierarchical learningStep 3 – initialize hierarchical procedural memoryStep 4 – demonstrate learning from interactionsStep 5 – add strategy retrieval and performance feedbackStep 5a – demonstrate hierarchical retrievalStep 5b – demonstrate performance feedback loopStep 5c – visualize procedural memory structureStep 5d – test edge cases and fallbacksArchitectural implications and production readinessSummarySubscribe for a free eBook
Advanced RAG with Complete Memory Integration
Technical requirementsCode lab 19.1 – creating a complete CoALA agent with integrated memory systemsStep 1 – create the complete agent with all memory typesStep 2 – load synthetic investment dataStep 3 – process baseline conversations for initial learningStep 4 – test improved performance and trigger adaptationsStep 5 – complete learning progression and hierarchical retrievalLangMem’s learning algorithms: choosing the right approachprompt_memory: efficient single-pass learninggradient: critique and proposal separationmetaprompt: multi-stage reflection for complex patternsChoosing the right algorithmCombining algorithms for comprehensive learningHow multi-algorithm learning works in practiceHands-on explorationDesigning domain metrics: the art of measuring successSingle goal optimization: clarity with consequencesMulti-goal balancing: realism with complexityConflict resolution through weightsMetric design patternsEvolution over timePractical recommendationsAdapting to your domain: from an investment advisor to any expert systemDomain conversion frameworkConversion strategy with LLMsDirectory structure for new domainPutting it all together: the complete CoALA architectureIntegration: four memory types working togetherPractical implications: enabling rapid innovationSummaryGet This Book’s PDF Version and Exclusive Extras
Unlock Your Exclusive Benefits
Unlock this Book’s Free Benefits in 3 Easy StepsStep 1Step 2Step 3Need help?
Other Books You May Enjoy
Join our Discord and Reddit space
Index

Content preview from Unlocking Data with Generative AI and RAG - Second Edition

9 Evaluating RAG Quantitatively and with Visualizations

Evaluation plays a crucial role in building and maintaining retrieval-augmented generation (RAG) pipelines. While you build the pipeline, you can use evaluation to identify areas for improvement, optimize the system’s performance, and systematically measure the impact of improvements. When your RAG system is deployed, evaluation can help ensure the effectiveness, reliability, and performance of the system.

In this chapter, we will cover the following topics:

Evaluating when building a RAG application
Evaluating a RAG application after deployment
Standardized evaluation frameworks
Ground truth
Code lab 9.1 – ragas
Additional evaluation techniques for RAG systems

Let’s start by talking ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Unlocking Data with Generative AI and RAG

Publisher Resources

ISBN: 9781806381654

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Unlocking Data with Generative AI and RAG - Second Edition

by Keith Bourne

9

Evaluating RAG Quantitatively and with Visualizations

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.