book

Agentic AI for Engineers: Architecting Goal-Driven Systems

Name: Agentic AI for Engineers: Architecting Goal-Driven Systems
Author: Dhivya Nagasubramanian
ISBN: 9798868823619

by Dhivya Nagasubramanian

March 2026

Intermediate

460 pages

10h 4m

English

Apress

Read now

Unlock full access

Agentic AI for Engineers
Foreword
Introduction
Who This Book Is ForHow to Read This BookWhat’s Ahead
Acknowledgments
Table of Contents
About the Author
About the Technical Reviewer
Part I: Artificial Intelligence-Foundation
1. Introduction: AI and Evolution of Agentic AI
The Many Minds of AI—Exploring Its Key BranchesMachine Learning—The Learner of PatternsSupervised Learning: The Flashcard LearnerUnsupervised Learning: The Curious ExplorerReinforcement Learning: The Trial-and-Error LearnerNatural Language Processing—The Language InterpreterSpeech and Audio—Giving Voice to AIComputer Vision—The Eyes of AIDeep Learning—The Architect of ComplexityWhy It Matters for Agentic AITransformers: A New Language for MachinesGenerative AI: Machines That CreateAgentic AI—The Doers and DecidersThe Shift: From Static to StrategicThe Agentic AI Engine: A Four-Step Intelligence CycleFrom Tools to TeammatesEngineering the FutureWhere Does Generative AI Fit in the Hierarchy of AI?The Parallel Path: A History of AutomationFrom Rules to Routines: The Early Days of Automation (2000 and Prior)RPA: Software That Mimicked Us (Early 2000s)Intelligent Automation: The Merge of Automation and AI (Mid-2010s–2020)Autonomy at Last: Enter Agentic AI (2023 and Beyond)A Shift in PhilosophyThe Evolution of Agentic AI: From Concept to RealityScenarioAI Agents—A Digital Coworker - Sneak Peek into Real-World ApplicationsKey Takeaways
2. Automation to Autonomy: A New Engineering Mindshift
AI Agents Leading to an Autonomous System Is a ProgressionThe Engineering Mindset for Automation: Rule-Based and DeterministicEfficiency Through Explicit LogicReliability, Traceability, and TimingReal-World ExamplesThe Common ThreadEngineering Mindset for AutonomyAutonomy As Goal-Directed BehaviorEmbracing Uncertainty As a Design PrincipleAutomation vs. Autonomy: Two Mindsets, Same ProblemExample 1: Warehouse Inventory RobotExample 2: Retail Order FulfillmentEnabling Technologies Behind the ShiftLarge Language Models and the Transformer RevolutionInstruction Tuning: From Text Generation to Goal FollowingPerception-Action Loops: Closing the Sense-Act GapMemory Systems: Enabling Continuity and LearningTool Use and Orchestration FrameworksThe ConvergenceMore Than Just Building: The Responsibility of Agentic AIA Manifesto for Responsible Agentic AI EngineeringCall to Action for EngineersChoosing RPA vs. Agents by Control and ComplexityKey Takeaways

3. Transformer Models and LLM Architecture
From Sequence Models to TransformersUnlocking Flexible CognitionThe Agentic Leap: Reasoning, Memory, PlanningTimeline: From RNNs to Transformers to Agentic AI1997–2014: Foundations in Recurrent Neural Networks2015–2016: RNN Enhancements and Early Attention Mechanisms2017: The Breakthrough2018: Decoder-Only Models Emerge2019–2020: Scaling Transformers2021: Multi-step Planning Foundations2022: Instruction Tuning and Emergent Behaviors2023–2024: Planning, Memory, and Modularity2025: Agentic AI Goes MainstreamWhat Makes Up a Transformer and Its Role in Agentic AIFrom Text to Tokens: How Machines ReadWhy Not Just Use Words?Why Not Just Use Characters?The Solution: Subword TokenizationThe GPS of Language: How Embeddings Guide TransformersWhy Embeddings MatterWhy Self-Attention and Positional Encoding Matter for Agentic AIPositional Encoding: Sequencing Without RecurrenceHow Positional Encoding WorksDifferent Architectures of Transformers for Different PurposesPretraining Strategies for LLMsScaling Laws and Emergence: When Bigger Becomes SmarterThe Model’s Memory Span: Understanding Context WindowsPros and Cons of Bigger vs. Smaller Context WindowsAdvantages of Bigger Context WindowsDisadvantages of Bigger Context WindowsEmergence: When Bigger Means Different, Not Just BetterExamples of Emergent CapabilitiesWhy This Matters for Agentic AIFrom Transformers to Cognitive EnginesFrom Prediction to SimulationPrompting As the Interface to CognitionCognitive Engines in ActionAgent Architectures Powered by LLMsHow Transformers Are Embedded in AgentsThe Autonomy Loop: Observe ➤ Reflect ➤ ActExample: Calendar Scheduling AgentCommon Challenges of Large Language ModelsLooking AheadKey Takeaways
4. The Agentic AI Fundamentals: Goals, Environments, Actions
Let’s Get the Definitions ClearAgency Begins with IntentWhat Makes Up an Agentic AI System?Defining and Structuring Goals: The Compass of AgentsWhat Is a Goal? (Tasks vs. Missions vs. Objectives)Real-World Use CaseStatic vs. Evolving GoalsHierarchical Goal Structures: From Macro to MicroReactive vs. Proactive AgentsUnderstanding Environments: Observability and UncertaintyWhy Environment Understanding MattersAction: From Intent to ImpactSo, What Is Action in Agentic AI?So, What Exactly Is a Task?—The Small Steps Behind Big Missions“Beyond Language: How Tools Empower Agents”The Tool Selection ProcessHow LLMs Choose ToolsError Handling When Tools Fail: A Comprehensive GuideFailure CategoriesTool Composition and ChainingCustom Tool CreationLoop Prevention and RecoverySubgoal Conflict ResolutionThe Invisible Thread That Holds Everything TogetherWhy Memory MattersAgentic Memory Types with Business ExamplesHow Agents Use Memory to AdaptDesign Considerations: What to Store and WhenBeyond Everything, What Enables Agentic AI for What It Is?—The ReasoningCoordination Between AgentsRule of Thumb: One Agent One SubgoalStochastic Brains: Embracing Variability in LLMsDesigning Guardrails for Agentic AIKey Takeaways for Engineers
Part II: Building Blocks of Agentic Systems
5. Architectural Design Patterns for Agentic Systems
Two Levels of Agentic DesignPatterns in AgentsThe Single-Agent Loop: Minimalist AutonomyThe Planner–Executor–Reflector Pattern—“Think Ahead, then Execute.”Tool-Augmented Reasoning Agents = Embedded Reasoning + External ActionExample: ReAct + Tool Frameworks in PracticeExample of Tool-Augmented Reasoning with RAGDesign Topologies in Agentic Architectures—How Multiple Agents Are WiredSequential Example—Retail Support Agent PipelineExample of Hierarchical Topology—AI NewsletterExample of Hybrid Topology—Universal Enterprise AssistantPayroll: Structured Change Under GovernanceHealth: Policy-Driven PrecisionTech: Autonomous Service with GuardrailsController and Coordination LayerConcurrent/Parallel Multi-agent PatternChoosing Between Single Agent, PER, and Multi-agent ArchitecturesDecision Tree on Picking the Right Agentic PatternChallenges and ConsiderationsMake Tools Contract First—and Enforce the Contract Both WaysReduce Hallucination at the Two Choke Points: Tool Choice and ArgumentsPut Guardrails Outside the Model—Least Privilege, Action Limits, and ApprovalsRetries Should Be Purposeful: Classify, Back Off, and Avoid Duplicate Side EffectsBound the Loop and Budget; Detect “Infinite” or Unproductive IterationsTeach the Model to Reflect When StuckPrefer Smaller, Specialized Models and Selective Tool UseHarden Untrusted Input and BrowsingOperationalizing Guardrails and Scaling TogetherKey Takeaways
6. The Art of Prompting
Prompting As Agent ProgrammingLLMs Don’t Use APIs—They Use LanguageAnalogy: Prompts As Dynamic Control PanelsCreating Effective PromptsPrompting Is Agent ProgrammingWhy Prompting Is Central to Reasoning, Planning, Safety, and DelegationHistory of PromptingPrompt Taxonomy: Forms and FunctionsComprehensive Prompt Taxonomy for Agentic AIPrompt Design As an Engineering DisciplineInstruction-Only Prompting: The Minimalist CommandFew-Shot Prompting: Learning by DemonstrationRole Prompting: Wearing the Right HatChain of Thought: Thinking Out LoudTree of Thought: Exploring the BranchesReAct: Reason, Act, ObserveReflexion: Learning From ItselfSelf-Ask Prompting: Questioning Before ActingPlanning Prompts: From Goal to Road MapDelegation Prompts: Assigning the WorkPrompt Chaining: Linking Thoughts TogetherGrounded Prompting: Anchoring to RealityEvaluation Prompts: The Critic’s EyeEnsemble Prompting: Many Minds, One AnswerAdaptive Prompting: Changing with the SituationContext Management Prompts: Staying FocusedPrompt Injection Resistance: Staying SafeTool-Specific Templates: Speaking the API’s LanguageMeta-Prompting: Thinking About ThinkingStructured Prompts for Consistent OutputsRole Framing and IterationPrompt Engineering As an Interface LayerPrompt Evaluation in the Agentic AI WorldWhy Testing Is Trickier for AgentsHow to Test in PracticeStep 1. Define ScenariosStep 2. Run the Full LoopStep 3. Score ItStep 4. Build in Self-CritiqueStep 5. Compare VariationsPrompt Caching: Cutting Costs and Latency at ScalePrompt Libraries: Managing Prompts As Production ArtifactsKey Takeaways
7. Tools and Frameworks for Building Agents
The New Ecosystem of Agentic AI FrameworksTool Layer—Hands and Eyes of an Agentic AI SystemTool Use: How Agents Reach Beyond ThemselvesWhy Tool Use MattersPractical ScenariosFoundation/Cognition Layer—Bedrock of an Agentic AI SystemThe Orchestration Layer—Where the Agent’s Mind Comes TogetherCloud Platforms for Building AgentsVisual Builder Layer—An Arena for Quick PrototypingKey Takeaways
8. Safety, Alignment, and Robustness in Agents
Hierarchy of Components in an Agentic AI EcosystemPatterns of Failure and MisalignmentAdversarial Attacks and Prompt InjectionDesigning Agentic Systems for SafetyConstraints and GuardrailsSimulation and SandboxingHuman-in-the-Loop vs. Human-on-the-LoopGoal Specification: The “What” and the “How”Robustness in Real-World ConditionsDistribution ShiftsStress TestingFail-Safe MechanismsAlignment StrategiesValue Alignment with Organizational and Human PrioritiesMeasuring Success: The Agent Evaluation FrameworkAgents Monitoring AgentWhat Monitoring Agents Actually Check ForPreventing the Monitor from Being GamedThe Performance and Cost RealityArchitectures That Work in PracticeEthical and Societal ConsiderationsFairness and Bias MitigationExample: Measuring Fairness in a Resume Screening AgentAccountability—Who’s Responsible?Privacy and Data Protection in Autonomous FlowsGlobal Standards: Cultural and Regulatory VariationPractical Techniques and ToolingGuardrails and Validation PipelinesObservability: Telemetry, Anomaly Detection, and LogsAdversarial and Stress Testing: PyRITTrust, Monitoring, and Responsible AI PlatformsCase Studies Across IndustriesDesigning for TrustObservability Through Structured LoggingThe Principle of Least PrivilegeModularity and Separation of ConcernsOpen ChallengesPre-deployment Safety ChecklistThe Cost of Safety: Engineering Trade-OffsKey Takeaways
Part III: Applications and Engineering Practice
9. Real-World Domain-Specific Use Cases of Agentic AI
Clarifying Terminology: AI Agent vs. Agentic AIThe Key DifferenceStacking Multiple Agents: AI Agents vs. Agentic AIMaking It Work: Strategy for Smooth ImplementationBuilding the Right Data InfrastructureExecutive Buy-In to the SolutionBusiness User’s AlignmentFinOps and Cost Optimization PlanningPilot Solution, Measure, and ScaleReal-World, Domain-Specific Wins with Agentic (and Agent-Like) AIHealthcareTriage and Diagnostic SupportClinical Workflow AutomationWhy Healthcare Moves CautiouslyThe Path ForwardFinanceCustomer-Facing Agents at ScaleAutonomous FinOpsRisk Modeling and Continuous Stress TestingCompliance MonitoringWhat Finance Teaches UsWhat Can Go Wrong: Speed Amplifies ErrorsGuardrails for the Highly Regulated IndustryManufacturing and Supply ChainManufacturing: Optimizing for the Wrong ThingCustomer ExperienceCustomer Service: Trust Is FragileCybersecurityOffensive Security TestingSmarter Case ManagementCybersecurity: False Positives Disrupt BusinessSmart GovernanceClimate and SustainabilityPublic Sector: Bias Has Democratic ConsequencesHow to Identify and Implement Agentic AI Use CasesKey TakeawaysThe Evolution of AI to Agentic SystemsDomain-Specific InsightsRisks and Failure ModesImplementation StrategyIdentifying Where Agentic AI FitsThe Bigger Picture
10. Build Your First AI Agent—Hands-On Coding
Why Start SmallThe Problem We’ll SolveSetting Up Your EnvironmentPrerequisitesUnderstanding CrewAI: The Core ConceptsAgentsTasksToolsCrewsBuilding Our First Agent—Personalized Research AssistantStep 1: A Single Research AgentStep 2: A Multi-agent PipelineStep 3: Iterative Refinement with Quality ThresholdsStep 4: Adding a Judge LLMStep 4: Human-in-the-Loop ApprovalStep 5: Adding GuardrailsKey TakeawaysWhat’s Next
11. Engineering Agent Feedback Loops
Agents Without Feedback = Static, Brittle SystemsFeedback → Adaptation → TrustBalances Autonomy with AccountabilityTypes of Feedback LoopsSelf-CritiqueImplicit Task FeedbackStep 1—Define the Feedback SignalsStep 2—Instrument Everything by Building Telemetry LayerStep 3—Translate Telemetry into Reward FunctionsStep 4—Build the Feedback LoopStep 5—Add Guardrails and an Exploration BudgetMechanisms for Feedback CollectionDesigning Feedback IntegrationImmediate CorrectionPractical TechniquesExample—Research AssistantChallenges and PitfallsChoosing the Right Feedback MechanismsThe Core Trade-OffsDecision Framework by Resource ConstraintsDecision Framework by Failure ModeCombining Mechanisms: Layered Feedback ArchitectureMonitoring Feedback Loop HealthFeedback Collection MetricsLearning Effectiveness MetricsFeedback Loop Latency MetricsBalance and Stability MetricsPutting It All Together: A Feedback Loop Health ScorecardKey Takeaways
12. Collaborative Agents (Multi-agent Systems, Human–AI Teaming)
Multi-agent System PatternsHands-On Architecture GuideArchitecture 1: Sequential PipelineArchitecture 2: Parallel Fan-Out/Fan-InArchitecture 3: Hierarchical (Manager-Worker)Architecture 4: Debate/AdversarialArchitecture 5: Voting/ConsensusArchitecture 6: Blackboard (Shared Memory)Architecture 7: Market-Based (Auction)Architecture 8: Supervisor with Dynamic RoutingCommunication and Coordination Is Central to SuccessProtocols: Messaging, Shared Memory, Blackboard SystemsTool Routing: Who Does What?Context Sharing Memory SynchronizationTiming: Sequential vs. Parallel Task ExecutionHuman–AI TeamingHuman in the LoopHuman on the LoopAgents As CopilotsExample Use CasesFinance: Trading Desk CopilotsHealthcare: Diagnostic Support TeamsSupply Chain: Negotiation + Routing AgentsSustainability: Swarm Agents on the GridDesign ChallengesEmergent BehaviorsConflict ResolutionBottlenecksError CascadesSecurity RisksMemory Architecture and Context Window ConstraintsToken Budget AllocationTesting Strategies for Multi-agent SystemsTesting for Emergent BehaviorsOptimization StrategiesWhen to Add More AgentsKey Takeaways
13. Testing, Debugging, Evaluation, and Deployment Considerations
Why Agents Aren’t Just Software: A New Mindset for DeploymentTesting Dynamic BehaviorDebugging Reasoning, Not Just CodeDeployment As Continuous EvolutionWhat Can Go Wrong with Agentic Systems—and How to Get ReadyTesting Agent BehaviorScenario-Based TestsReasoning ValidationPrompt Regression TestingAdversarial TestingOperational Trust: Observability in Agentic AITelemetry PipelinesDashboardsUser Feedback LoopsWhy Evaluation Matters in Agentic AIThe Dimensions of EvaluationCurrent Frameworks and AdaptationsCase Study: Precision Under Pressure—Evaluating Clinical Decision Support AgentsHow We Evaluate in PracticePutting It into Practice: A Prompt Evaluation ExperimentExperiment SetupEvaluating the OutputsThe Unfinished Science of Evaluating AgentsWhere Evaluation Is HeadedDeployment Best PracticesContainerizationSecrets and Configuration ManagementCI/CD for AgentsCanary Releases and A/B TestingFailure PlanningLessons from Real-World DeploymentsSilent FailuresPrompt Drift Across Model UpdatesBalancing Observability and PrivacyOperational Cost of Continuous MonitoringKey TakeawaysWhy Testing, Evaluation, and Deployment MatterTesting Agent BehaviorDebugging AgentsEvaluation FrameworksObservability and MonitoringDeployment Best PracticesPitfalls and ChallengesThe Future of Evaluation
14. Conclusion and the Road Ahead: Open Challenges and Responsibility
Grand Technical Hurdles and Open Research QuestionsGovernance and Alignment ChallengesResponsibility and AccountabilityAlignment at ScaleCultural and Legal VariationTransparency and TrustEthical and Societal QuestionsWhat Comes Next: Emerging Horizons in Agentic AIMeta-agentsAgent SimulationsNeuro-symbolic HybridsHuman–AI Intent FusionA Responsible Path ForwardKey Takeaways for EngineersKey TakeawaysSynthesis of the JourneyOpen Technical ChallengesGovernance and AlignmentEthical ResponsibilitiesRegulatory LandscapeEmerging FrontiersThe Responsible Path Forward
Index

Content preview from Agentic AI for Engineers: Architecting Goal-Driven Systems

About the Technical Reviewer

Jayanta Sen

is a seasoned enterprise AI and data engineering leader and architect with over 19 years of experience across strategy, architecture, consulting, technical program and product management, data engineering, and AI/ML. He blends deep technical proficiency with practical business insight, helping organizations build and scale AI-driven data platforms, AI and data strategy and governance frameworks, agents, and intelligent business processes.

Currently, he serves as a Principal Solution Engineer at Microsoft, where he leads global customer AI solution engineering focusing on agentic AI, machine learning, intelligent ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9798868823619Purchase Link Publisher Website

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Agentic AI for Engineers: Architecting Goal-Driven Systems

by Dhivya Nagasubramanian

About the Technical Reviewer

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.