book

Building Data-Driven Applications with LlamaIndex - Second Edition

Name: Building Data-Driven Applications with LlamaIndex - Second Edition
Author: Andrei Gheorghiu
ISBN: 9781806021857

by Andrei Gheorghiu

May 2026

Intermediate

640 pages

16h 47m

English

Packt Publishing

Read now

Unlock full access

Preface
Free benefits with your book
Part 1: Introduction to Generative AI and LlamaIndex
Chapter 1: Understanding Large Language Models
Introducing GenAI and LLMsWhat is GenAI?What is an LLM?Understanding the role of LLMs in modern technologyExploring challenges with LLMsNo native access to real-time dataNo built-in sense of truth or accuracyContext limitations and memory lossImproper reasoningLack of transparency and explainabilitySustainability and environmental impactSynthetic text and data contaminationAugmenting LLMs with RAGSummary
Chapter 2: LlamaIndex: The Hidden Jewel – An Introduction to the LlamaIndex Ecosystem
Technical requirementsOptimizing language models – an overview of popular approachesLLM fine-tuningPrompt engineeringKnowledge distillationTool use and function callingWhat LlamaIndex doesMaking the best use of this bookIntroducing our hands-on project: Contract Review ExpertPreparing our coding environmentInstalling PythonInstalling GitInstalling LlamaIndexSetting up OllamaPulling required modelsStarting OllamaDiscovering Streamlit – the perfect tool for rapid building and deployment!One final checkFamiliarizing ourselves with the structure of the LlamaIndex code repositorySummary
Part 2: Starting Your First LlamaIndex Project
Chapter 3: Kickstarting Your Journey with LlamaIndex
Technical requirementsUncovering the essential building blocks of LlamaIndex – documents, nodes, and indexesDocumentsNodesThe BaseNode and Node classes and their subtypesUnderstanding the TextNode class and its usesManually creating TextNode objectsAutomatically extracting nodes from documents using splittersNodes don't like to be alone – they crave relationshipsIndexesA quick recap of the key conceptsBuilding our first interactive, augmented LLM applicationUsing logging to better understand the logic and debug our applicationsCustomizing the AI models used by LlamaIndexBuilding a simple AI agentStarting our hands-on exerciseSummary
Chapter 4: Ingesting Data into Our RAG Workflow
Technical requirementsIngesting data via LlamaHubAn overview of LlamaHubUsing the LlamaHub data loaders to ingest contentIngesting data from a web pageIngesting data from a databaseBulk-ingesting data from sources with multiple file formatsParsing documents into nodesUnderstanding simple text splittersSentenceSplitterTokenTextSplitterCodeSplitterUsing more advanced node parsersSentenceWindowNodeParserLangchainNodeParserSimpleFileNodeParserHTMLNodeParserMarkdownNodeParserJSONNodeParserSemanticSplitNodeParserUsing relational parsersHierarchicalNodeParserUnstructuredElementNodeParserUnderstanding chunk_size and chunk_overlapIncluding relationships with include_prev_next_relPractical ways of using text splitters and node parsersOption 1: Standalone usageOption 2: Configured in settingsOption 3: As part of an ingestion pipelineWorking with metadata to improve the contextSummaryExtractorQuestionsAnsweredExtractorTitleExtractorEntityExtractorKeywordExtractorPydanticProgramExtractorMarvinMetadataExtractorDefining your custom extractorIs having all that metadata always a good thing?Estimating the potential cost of using metadata extractorsPreserving data privacyScrubbing personal data and other sensitive informationUsing the NERPIINodePostprocessorUsing the PresidioPIINodePostprocessorUsing the ingestion pipeline to increase efficiencyHands-on – ingesting documents into our Contract Review Expert applicationSummary
Chapter 5: Indexing with LlamaIndex
Technical requirementsIndexing data – a bird's-eye viewUnderstanding VectorStoreIndexConstructing a vector store indexUnderstanding embeddingsUnderstanding similarity searchCosine similarityDot productEuclidean distanceUnderstanding how embeddings are integrated into LlamaIndexHow do I decide which embedding model to use?Persisting and reusing indexesUnderstanding StorageContextThe difference between vector stores and vector databasesExploring other index types in LlamaIndexSummaryIndexUnderstanding the inner workings of SummaryIndexConstructing a summary indexPractical use caseDocumentSummaryIndexUnderstanding the inner workings of DocumentSummaryIndexConstructing a document summary indexPractical use caseKeywordTableIndexUnderstanding the inner workings of the KeywordTableIndexConstructing a keyword table indexPractical use caseTreeIndexUnderstanding the inner workings of the TreeIndexConstructing a tree indexPractical use casePotential drawbacks of using the TreeIndexPropertyGraphIndexUnderstanding the inner workings of PropertyGraphIndexConstructing a property graph indexPractical use caseMultiModalVectorStoreIndexConstructing a multimodal vector store indexPractical use caseChoosing the right index for your use caseBuilding indexes on top of other indexes with ComposableGraphHow to use ComposableGraphA more detailed description of this conceptManaging the lifecycle of an indexSetting everything up for efficient index updatesInserting new documents into the indexUpdating existing documentsDeleting documents from the indexAutomatically refreshing an indexImplementing versioningBest practicesCost considerations when building and querying indexesSummary
Part 3: Retrieving and Working with Indexed Data
Chapter 6: Querying Our Data, Part 1 – Context Retrieval
Technical requirementsLearning about query mechanics – an overviewUnderstanding the basic retrieversThe VectorStoreIndex retrieversVectorIndexRetrieverVectorIndexAutoRetrieverThe SummaryIndex retrieversSummaryIndexRetrieverSummaryIndexEmbeddingRetrieverSummaryIndexLLMRetrieverThe DocumentSummaryIndex retrieversDocumentSummaryIndexLLMRetrieverDocumentSummaryIndexEmbeddingRetrieverThe TreeIndex retrieversTreeSelectLeafRetrieverTreeSelectLeafEmbeddingRetrieverTreeAllLeafRetrieverTreeRootRetrieverThe KeywordTableIndex retrieversKeywordTableGPTRetrieverKeywordTableSimpleRetrieverKeywordTableRAKERetrieverThe PropertyGraphIndex retrieversBehavior of the default PGRetrieverCustomizing the PropertyGraphIndex retrieversUsing the Cypher-based sub-retrieversCommon characteristics shared by all retrieversEfficient use of retrieval mechanisms – asynchronous operationBuilding more advanced retrieval mechanismsThe naïve retrieval methodImplementing metadata filtersUsing selectors for more advanced decision logicUnderstanding toolsTransforming and rewriting queriesCreating more specific sub-queriesUnderstanding the concepts of dense and sparse retrievalDense retrievalSparse retrievalTF-IDF in sparse retrievalImplementing sparse retrieval in LlamaIndexWhen should we use sparse retrieval instead of dense retrieval?When would dense retrieval be a better choice?Can we combine the two methods in a single retriever?Discovering other advanced retrieval methodsDebugging common retrieval problemsDiagnosing irrelevant nodesDiagnosing metadata filter problemsDiagnosing empty or weak resultsMoving from ad hoc debugging to systematic debuggingSummary

Chapter 7: Querying Our Data, Part 2 – Postprocessing and Response Synthesis
Technical requirementsRe-ranking, transforming, and filtering nodes using postprocessorsNode filtering postprocessorsSimilarityPostprocessorKeywordNodePostprocessorPIINodePostprocessor and NERPIINodePostprocessorNode transforming postprocessorsMetadataReplacementPostprocessorSentenceEmbeddingOptimizerLongLLMLinguaPostprocessorNode re-ranking postprocessorsLLMRerankStructuredLLMRerankCohereRerankSentenceTransformerRerankRankGPTRerankLongContextReorderTime-based postprocessorsAdditional re-rankers not included in the LlamaIndex coreMeasuring the effectiveness of LLM-based re-rankingUnderstanding the model drift phenomenonContext augmentation postprocessorsPrevNextNodePostprocessor and AutoPrevNextNodePostprocessorFinal thoughts about node postprocessorsUnderstanding response synthesizersExtracting structured outputsUnderstanding the purpose of PydanticGenerating structured outputs using structured LLMsExtracting structured outputs using output parsersGuardrailsOutputParserLangchainOutputParserExtracting structured outputs using Pydantic programsBuilding and using query enginesExploring different methods of building query enginesAdvanced uses of the QueryEngine interfaceImplementing advanced routing with RouterQueryEngineQuerying multiple documents with SubQuestionQueryEngineSummary
Chapter 8: Building Faster and Smarter with Workflows
Technical requirementsUnderstanding workflows and their use casesBuilding a simple workflowUnderstanding the core elements of a workflowThe Workflow classThe stepsThe eventsThe contextUsing advanced workflow elementsImplementing checkpoints and retriesImplementing branches and loopsImplementing human-in-the-loop logicDesigning efficient workflow architecturesUnderstanding the common architectural patterns of a workflowDesigning for performance and scaleKeeping workflows maintainableBuilding resilience into the designAdding workflows to our hands-on projectThe contract_analysis_workflow.py moduleThe contracts.py moduleSummary
Part 4: Building Chatbots and Agents with LlamaIndex
Chapter 9: Understanding Basic Agent Architectures
Technical requirementsFrom chatbots to RAG-driven conversationsDiscovering ChatEngineUnderstanding how the chat engine maintains conversation historyUnderstanding the different chat modesSimple modeContext modeCondense question modeCondense and context modeImplementing agentic strategies in our appsUnderstanding how agent chat memory worksUnderstanding how short-term memory worksUnderstanding how long-term memory is implementedWhere is memory stored by default?Some recommended best practices when implementing memoryBuilding tools and ToolSpec classes for our agentsUnderstanding reasoning loopsFunctionAgentReActAgentHands-on: Implementing conversation tracking for Contract Review ExpertSummary
Chapter 10: Exploring Advanced Agentic Use Cases
Technical requirementsFrom workflows to agentsHow LlamaIndex turns tools, memory, and models into agentsHandling large tool outputs safelyUnderstanding the LoadAndSearchToolSpec utilityUnderstanding OnDemandLoaderToolScaling to many tools: tool retrieval and tool indexingShifting from a "tool list" to a "tool library"Tool indexing with ObjectIndexThe tool retriever: ObjectRetrieverHow retrieval fits into the agent loopPractical guidance for large tool librariesOrchestrating agents with AgentWorkflowHandoffs are the whole gameA small set of orchestration patterns covers most real systemsHuman-in-the-loop orchestrationGuardrails that keep orchestration from turning into chaosPutting it all together: A multi-agent exampleSummary
Part 5: Customization, Evaluation, Prompt Engineering, and Final Words
Chapter 11: Customizing and Deploying Our LlamaIndex Project
Technical requirementsCustomizing our RAG componentsHow the LLaMA models changed the open source landscapeRunning a local LLM using LM StudioUnderstanding LLM quantizationSo, which model should you choose?So far, so good, but where's the RAG part in all this?Routing between LLMs using services such as NeutrinoLeveraging the plug-and-play convenience of using Llama PacksUsing the Llama CLIHow RAG works on the command lineDeploying applications with StreamlitDeploying the Contract Review Expert to Streamlit Community CloudSummary
Chapter 12: Tuning Our RAG Implementation – Observability and Evaluation
Technical requirementsWhy does RAG evaluation matter?What are observability and evaluation?Two sides of the same coinTracing RAG workflows with LlamaIndex's instrumentation moduleTracing a simple RAG queryReading and interpreting trace outputCapturing events for deeper insightsOpenTelemetry and production observabilityMeasuring RAG quality - key concepts and metricsThe two dimensions of RAG evaluationKey evaluation metricsRetrieval metrics: hit rate and MRRThe LLM-as-a-judge patternEvaluating with LlamaIndex's built-in modulesRunning individual evaluators in the evaluation moduleEvaluating faithfulness, relevancy, and correctnessGenerating a synthetic evaluation datasetUsing BatchEvalRunner for efficiencyInterpreting results and taking actionBuilding a simple evaluation pipelineSummary
Chapter 13: Prompt Engineering Guidelines and Best Practices
Technical requirementsWhy prompts are your secret weaponUnderstanding how LlamaIndex uses promptsRichPromptTemplate – the modern approach to prompt templatesInspecting the prompts used by RAG componentsCustomizing default promptsThe golden rules of prompt engineeringAccuracy and clarity of expressionDirectivenessContext qualityContext quantityContext orderingRequired output formatInference costOverall system latencyChoosing the right LLM for the taskModel architectureModel sizeInference speedChat modelsInstruct modelsCoding modelsSummarization modelsTranslation modelsQuestion answering modelsCommon methods used for creating effective promptsFew-shot prompting, also known as k-shot promptingChain-of-thought (CoT) promptingSelf-consistencyTree of thoughts (ToT) promptingPrompt chainingEmerging trends in promptingFine-tuning RAG components for better resultsFine-tuning embedding modelsFine-tuning cross-encoders for re-rankingFine-tuning the LLM for better response synthesisFine-tuning an evaluatorDeciding when to fine-tuneExploring cost optimization techniquesChoosing the right model for the taskOptimizing token usageCaching strategiesLeveraging open source modelsMonitoring and tracking costsSummary
Chapter 14: Going Enterprise with LlamaParse
Technical requirementsDiscovering the features and advantages provided by LlamaParseThe platform at a glanceThe credit system and getting startedHow LlamaParse integrates with LlamaIndexSecurity, privacy, and complianceCurrent limitations and the question of vendor lock-inUsing LlamaParse for state-of-the-art data ingestionLlamaParse v2 and the tier systemHands-on: parsing a document with LlamaParseIntegrating LlamaParse into a LlamaIndex ingestion pipelineCustomizing LlamaParse output with promptsUsing LiteParse – the open source alternativeHands-on: parsing a document with LiteParseWhen to use whichWhen to go enterpriseWhen open source still meets your needsMitigating vendor lock-in: a practical checklistCost planning for productionSummary
Chapter 15: Conclusion and Additional Resources
Technical requirementsOther projects and further learningThe LlamaIndex example collectionFull-stack projects and LlamaHubThe power of many – the LlamaIndex communityKey takeaways, final words, and encouragementFuture RAG trends in the larger context of generative AITrend 1: The "RAG is dead" debate has settledTrend 2: The Model Context Protocol is becoming standardizedTrend 3: Specialized inference hardware has become its own categoryTrend 4: Multimodal is no longer emergingTrend 5: The AI regulation landscape is hereA small philosophical nugget for you to considerSummary
Chapter 16: Unlock Your Exclusive Benefits
Unlock this Book's Free Benefits in 3 Easy Steps
Index

Content preview from Building Data-Driven Applications with LlamaIndex - Second Edition

9 Understanding Basic Agent Architectures

In Chapter 3, we saw an example of an AI agent that can use tools to respond to user queries. But that was more of a teaser meant to demonstrate how easily LlamaIndex allows us to build agentic systems. Now it's time to explore in detail how agents work and the different design paradigms available to us.

In this chapter, we will explore the various chat engine modes available, from simple chatbots to more advanced context-aware and question-condensing engines. Then, we'll compare different agent architectures, analyzing tools, reasoning loops, and parallel execution methods. Understanding these different modes matters because choosing the wrong chat architecture can mean the difference between a chatbot ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Building Data-Driven Applications with LlamaIndex

Publisher Resources

ISBN: 9781806021857

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Building Data-Driven Applications with LlamaIndex - Second Edition

by Andrei Gheorghiu

9

Understanding Basic Agent Architectures

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.