book

Unlocking Data with Generative AI and RAG

Name: Unlocking Data with Generative AI and RAG
Author: Keith Bourne
ISBN: 9781835887905

by Keith Bourne

September 2024

Intermediate to advanced

350 pages

10h 32m

English

Packt Publishing

Read now

Unlock full access

Unlocking Data with Generative AI and RAG
Foreword
Contributors
About the author
About the reviewers
Preface
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesConventions usedGet in touchShare Your ThoughtsDownload a free PDF copy of this book
Part 1 – Introduction to Retrieval-Augmented Generation (RAG)
Chapter 1: What Is Retrieval-Augmented Generation (RAG)
Understanding RAG – Basics and principlesAdvantages of RAGChallenges of RAGRAG vocabularyLLMPrompting, prompt design, and prompt engineeringLangChain and LlamaIndexInferenceContext windowFine-tuning – full-model fine-tuning (FMFT) and parameter-efficient fine-tuning (PEFT)Vector store or vector database?Vectors, vectors, vectors!VectorsImplementing RAG in AI applicationsComparing RAG with conventional generative AIComparing RAG with model fine-tuningThe architecture of RAG systemsSummary
Chapter 2: Code Lab – An Entire RAG Pipeline
Technical requirementsNo interface!Setting up a large language model (LLM) accountInstalling the necessary packagesImportsOpenAI connectionIndexingWeb loading and crawlingSplittingEmbedding and indexing the chunksRetrieval and generationPrompt templates from the LangChain HubFormatting a function so that it matches the next step’s inputDefining your LLMSetting up a LangChain chain using LCELSubmitting a question for RAGFinal outputComplete codeSummary
Chapter 3: Practical Applications of RAG
Technical requirementsCustomer support and chatbots with RAGTechnical supportFinancial servicesHealthcareRAG for automated reportingHow RAG is utilized with automated reportingTransforming unstructured data into actionable insightsEnhancing decision-making and strategic planningE-commerce supportDynamic online product descriptionsProduct recommendations for e-commerce sitesUtilizing knowledge bases with RAGSearchability and utility of internal knowledge basesExpanding and enhancing private data with general external knowledge basesInnovation scouting and trend analysisLeveraging RAG for personalized recommendations in marketing communicationsTraining and educationCode lab 3.1 – Adding sources to your RAGSummary
Chapter 4: Components of a RAG System
Technical requirementsKey component overviewIndexingRetrieval and generationRetrieval focused stepsGeneration stagePromptingDefining your LLMUIPre-processingPost-processingOutput interfaceEvaluationSummaryReferences

Chapter 5: Managing Security in RAG Applications
Technical requirementsHow RAG can be leveraged as a security solutionLimiting dataEnsuring the reliability of generated contentMaintaining transparencyRAG security challengesLLMs as black boxesPrivacy concerns and protecting user dataHallucinationsRed teamingCommon areas to target with red teamingResources for building your red team planCode lab 5.1 – Securing your keysCode lab 5.2 – Red team attack!Code lab 5.3 – Blue team defend!Summary
Part 2 – Components of RAG
Chapter 6: Interfacing with RAG and Gradio
Technical requirementsWhy Gradio?Benefits of using GradioLimitations to using GradioCode lab – Adding a Gradio interfaceSummary
Chapter 7: The Key Role Vectors and Vector Stores Play in RAG
Technical requirementsFundamentals of vectors in RAGWhat is the difference between embeddings and vectors?What is a vector?Vector dimensions and sizeWhere vectors lurk in your codeVectorization occurs in two placesVector databases/stores store and contain vectorsVector similarity compares your vectorsThe amount of text you vectorize matters!Not all semantics are created equal!Code lab 7.1 – Common vectorization techniquesTerm frequency-inverse document frequency (TF-IDF)Word2Vec, Sentence2Vec, and Doc2VecBidirectional encoder representations from transformersOpenAI and other similar large-scale embedding servicesFactors in selecting a vectorization optionQuality of the embeddingCostNetwork availabilitySpeedEmbedding compatibilityGetting started with vector storesData sources (other than vector)Vector storesCommon vector store optionsChoosing a vector storeSummary
Chapter 8: Similarity Searching with Vectors
Technical requirementsDistance metrics versus similarity algorithms versus vector searchVector spaceSemantic versus keyword searchSemantic search exampleCode lab 8.1 – Semantic distance metricsEuclidean distance (L2)Dot product (also called inner product)Cosine distanceDifferent search paradigms – sparse, dense, and hybridDense searchSparse searchHybrid searchCode lab 8.2 – Hybrid search with a custom functionCode lab 8.3 – Hybrid search with LangChain’s EnsembleRetriever to replace our custom functionSemantic search algorithmsk-NNANNEnhancing search with indexing techniquesVector search optionspgvectorElasticsearchFAISSGoogle Vertex AI Vector SearchAzure AI SearchApproximate Nearest Neighbors Oh YeahPineconeWeaviateChromaSummary
Chapter 9: Evaluating RAG Quantitatively and with Visualizations
Technical requirementsEvaluate as you buildEvaluate after you deployEvaluation helps you get betterStandardized evaluation frameworksEmbedding model benchmarksVector store and vector search benchmarksLLM benchmarksFinal thoughts on standardized evaluation frameworksWhat is the ground truth?How to use the ground truth?Generating the ground truthHuman annotationExpert knowledgeCrowdsourcingSynthetic ground truthCode lab 9.1 – ragasSetting up LLMs/embedding modelsGenerating the synthetic ground truthAnalyzing the ragas resultsRetrieval evaluationGeneration evaluationEnd-to-end evaluationOther component-wise evaluationRagas founder insightsAdditional evaluation techniquesBilingual Evaluation Understudy (BLEU)Recall-Oriented Understudy for Gisting Evaluation (ROUGE)Semantic similarityHuman evaluationSummaryReferences
Chapter 10: Key RAG Components in LangChain
Technical requirementsCode lab 10.1 – LangChain vector storeVector stores, LangChain, and RAGCode lab 10.2 – LangChain RetrieversRetrievers, LangChain, and RAGCode lab 10.3 – LangChain LLMsLLMs, LangChain, and RAGExtending the LLM capabilitiesSummary
Chapter 11: Using LangChain to Get More from RAG
Technical requirementsCode lab 11.1 – Document loadersCode lab 11.2 – Text splittersCharacter text splitterRecursive character text splitterSemantic chunkerCode lab 11.3 – Output parsersString output parserJSON output parserSummary
Part 3 – Implementing Advanced RAG
Chapter 12: Combining RAG with the Power of AI Agents and LangGraph
Technical requirementsFundamentals of AI agents and RAG integrationLiving in an AI agent worldLLMs as the agents’ brainsGraphs, AI agents, and LangGraphCode lab 12.1 – adding a LangGraph agent to RAGTools and toolkitsAgent stateCore concepts of graph theoryNodes and edges in our agentCyclical graph setupSummary
Chapter 13: Using Prompt Engineering to Improve RAG Efforts
Technical requirementsPrompt parametersTemperatureTop-pSeedTake your shotPrompting, prompt design, and prompt engineering revisitedPrompt design versus engineering approachesFundamentals of prompt designAdapting prompts for different LLMsCode lab 13.1 – Custom prompt templateCode lab 13.2 – Prompting optionsIteratingIterating the toneShorten the lengthChanging the focusSummarizingSummarizing with a focusextract instead of summarizeInferenceExtracting key dataInferring topicsTransformationExpansionSummary
Chapter 14: Advanced RAG-Related Techniques for Improving Results
Technical requirementsNaïve RAG and its limitationsHybrid RAG/multi-vector RAG for improved retrievalRe-ranking in hybrid RAGCode lab 14.1 – Query expansionCode lab 14.2 – Query decompositionCode lab 14.3 – MM-RAGMulti-modalBenefits of multi-modalMulti-modal vector embeddingsImages are not just “pictures”Introducing MM-RAG in codeOther advanced RAG techniques to exploreIndexing improvementsRetrievalPost-retrieval/generationEntire RAG pipeline coverageSummary
Index
Why subscribe?
Other Books You May EnjoyPackt is searching for authors like youShare Your ThoughtsDownload a free PDF copy of this book

Content preview from Unlocking Data with Generative AI and RAG

Contributors

About the author

Keith Bourne is a senior Generative AI data scientist at Johnson & Johnson. He has over a decade of experience in machine learning and AI working across diverse projects in companies that range in size from start-ups to Fortune 500 companies. With an MBA from Babson College and a master’s in applied data science from the University of Michigan, he has developed several sophisticated modular Generative AI platforms from the ground up, using numerous advanced techniques, including RAG, AI agents, and foundational model fine-tuning. Keith seeks to share his knowledge with a broader audience, aiming to demystify the complexities of RAG for organizations looking to leverage this promising technology.

I would like to thank ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Building AI Agents with LLMs, RAG, and Knowledge Graphs

Publisher Resources

ISBN: 9781835887905

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Unlocking Data with Generative AI and RAG

by Keith Bourne

Contributors

About the author

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.