book

Building Agent-Powered Applications

Name: Building Agent-Powered Applications
Author: Vasyl Zvarydchuk
ISBN: 9781807605179

by Vasyl Zvarydchuk

April 2026

Intermediate

490 pages

14h 1m

English

Packt Publishing

Read now

Unlock full access

Preface
What this book is aboutWhat this book doesn't coverHow chapters fit togetherTerminology, index, and referencesFree benefits with your book
Part 1: Foundations of AI, NLP, and LLMs
Chapter 1: Artificial Intelligence and Natural Language Processing Fundamentals
Technical requirementsMachine learning basicsTraining machine learning modelsSupervised, unsupervised, and reinforcement learningNeural networksFine-tuningCore machine learning problemsRegressionClassificationClusteringRanking and recommendationCore NLP problems and conceptsTokenizationSentence-level tokenizationStemming, lemmatization, and stopwordsEmbeddingsGenerating embeddingsSentence-level embeddingsSimilarity metricsCosine similarityEuclidean distanceManhattan similaritySummary
Chapter 2: Understanding Large Language Models
Technical requirementsSequence-to-sequence modelsEncoderDecoderTransformersTrain, run, and benchmark LLMsTraining objectivesLLM training datasetsBenchmark LLMsSmall language modelsMultimodal modelsUsing LLMsContext windowSampling and randomnessHallucinationsLLM familiesGPT modelsGeminiLLaMAClaudeModel selectionSummary
Chapter 3: Prompt Engineering
Technical requirementsUnderstanding prompt structureSystem and user promptsSystem prompt structureUser prompt structureExamples and n-shot learningExample selectionZero-shot promptingStatic and dynamic few-shot promptingCore prompt techniquesChain of thoughtTree-of-Thought promptingSelf-consistency promptingSystem role or persona-based promptingOther techniquesMulti-prompt engineering techniquesStatic multi-prompt flowsDynamic multi-prompt flowsMulti-prompt chain of thoughtIterative self-refinementPrompt chainingSummary
Part 2: Building LLM Applications in Practice
Chapter 4: Understanding Language Tasks
Technical requirementsText summarizationExtractive summarizationAbstractive summarizationSummarization parametersSummary lengthMaximum tokensRandomness settingsRefocusModel selectionMultimodal summarizationText-first multimodal summarizationMultimodal modelsSummarizing tabular dataLarge document summarizationText classificationSentiment analysisIntent detectionInformation extractionNamed entity recognitionClassical NLP pipelinesTransformer-based fine-tuned modelsLarge language model for NERRelation and event extractionMultilanguage supportLanguage detectionMachine translationSummary
Chapter 5: Generation, Question Answering, and Reasoning
Text generationText generation parametersInstruction and promptFew-shot promptingOutput lengthRandomnessModel selectionCode generationSynthetic data generationQuestion answeringClosed-domain and open-domain QAMulti-hop and conversational QAQA for structured dataReasoningLLM reasoningCommonsense reasoningArithmetic, logical, and symbolic reasoningTemporal reasoningCausal reasoningAnalogical reasoningMulti-hop reasoningReasoning modelsUnknown reasoningSummary
Chapter 6: Retrieval-Augmented Generation
Technical requirementsRAG componentsStatic retrieval and parserGeneratorLexical retrievalTerm frequency and inverse document frequencyBM25Inverted indexesSemantic retrievalDocument preprocessingSelect document lengthEmbedding modelsChoose a similarity metricFind the most relevant documentsApproximate nearest neighbor (ANN)Tree-based methodsGraph-based methodsQuantization-based methodsRAG implementationsCustom pipelineLexical retrieverSemantic retrieverHybrid approach, vector databasesVector databasesRAG as a serviceSummary
Chapter 7: LLM Fine-Tuning
The anatomy of fine-tuningHow fine-tuning affects the modelFull fine-tuningPartial fine-tuningParameter-efficient fine-tuning: adapters and LoRASupervised fine-tuningPreference-based fine-tuning: RLHF and DPOWhen and how to fine-tuneHow to fine-tuneFine-tuning on local or on-premises hardwareFine-tuning in the cloudFine-tuning as a managed featureChoosing the right model and strategySelect the model familyChoosing the model sizeSelecting the fine-tuning strategyPrompting, RAG, or fine-tuningFine-tuning datasetsHosting fine-tuned modelsSummary

Part 3: Designing, Building, and Evaluating AI Agents
Chapter 8: Exploring the Architecture of AI Agents
Technical requirementsAgent basicsAgent terminologyLLM agent architectureConversational agentsBackground agentsLLM agent examplesCustomer support agentLead generation agentPersonal email filtering agentChat GPT, Gemini Chat, and Microsoft 365 CopilotAgent memoryShort-term memoryLLM context window and efficiencyContext relevanceLong-term memoryUser preferencesProfile dataPrevious conversationsTools and orchestrationAgent orchestrationOrchestration as a sequence of multi-label classificationOrchestration planMulti-plan orchestrationDynamic orchestration plansOrchestration challengesMulti-agent flowsCode-level multi-agent flowsMulti-agent communication patternsWorker and supervisorSequential chatRoundtable discussionHierarchical task delegation modelCompetitive agent modelAgent goals and control mechanismsSummary
Chapter 9: Building AI Agents
Technical requirementsAgent designFunctional requirementsSystem requirementsInteraction modeImplementation approachSkills, tools, and capabilitiesMemory, state, and learningReasoning and control mechanismsSafety and control boundariesEvaluation and success metricsDeployment and operationsAgent memory implementationCustom memory developmentMemory tools and frameworksMem0Agent platforms memory implementationMemory-augmented learningTools and orchestrationCustom tools orchestrationFunction calling frameworksOpenAI function calling frameworkLangChain function callingTools orchestration in an agentic platformModel context protocolAgent decision-making and planningAgent reasoning techniquesPlanning strategiesSequential decision-makingHierarchical task decompositionSkill chainingError handling and recovery strategiesHuman-in-the-loopMulti-agent frameworks and protocolsAutoGen framework (Microsoft Agent Framework)Agent-to-agent protocolImplementing AI agents in practiceEmail filtering agent – requirementsFunctional requirementsSystem requirementsInteraction modeMemory, state, and learningReasoning and control mechanismsSafety and control boundariesEvaluation and success metricsDeployment and operationsEmail filtering agent – designEmail filtering agent – implementationSummary
Chapter 10: Evaluating LLM Applications and Agents
Technical requirementsEvaluation challengesEvaluation strategyMulti-task AI agents' evaluationEvaluation datasetsPublic datasetsBuilding custom evaluation datasetsSynthetic and human-generated datasetsEvaluation metricsClosed-ended task evaluationAccuracy and exact matchPrecision, recall, and F1 for binary classificationMulti-class classification metricsSimilarity metrics for open-ended tasksLexical similarity-based metricsSemantic similarity-based metricsHuman as a judgeLLM as a judgeOnline vs.offline evaluationsAgent evaluationRAG-based agent evaluationRetriever evaluationGenerator evaluationConversational agent evaluationAgent memory evaluationTool orchestration evaluationBackground agent evaluationResponsible AIResponsible AI risks and evaluation criteriaEvaluation datasets for safety, bias, and fairnessJailbreak and adversarial prompt evaluationEvaluation tools and frameworksSummary
Chapter 11: Unlock Your Exclusive Benefits
Unlock this Book's Free Benefits in 3 Easy Steps
Appendix 1: Expert Insights
Appendix 2: Bibliography
Index

Content preview from Building Agent-Powered Applications

10 Evaluating LLM Applications and Agents

So, we've almost reached the end of our exciting journey through the world of AI, LLMs, and agents. Now it's time to turn to one of the most critical, yet often underestimated, aspects of the entire AI development lifecycle: evaluation.

You can design the most sophisticated LLM application or orchestrate the cleverest AI agent, but without proper evaluation, you're essentially flying "blind". You won't know if your agent is producing useful results, making harmful mistakes, or simply failing silently. Evaluation is how we gain visibility into the quality, reliability, and impact of what we've built and how we iterate and improve it.

In this final chapter, we will define what evaluation means in AI.

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781807605179

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Building Agent-Powered Applications

by Vasyl Zvarydchuk

10

Evaluating LLM Applications and Agents

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.