book

Practical LLM Evaluation for Production Systems

Name: Practical LLM Evaluation for Production Systems
ISBN: 9781807423896

by Ammar Mohanna, Indrajit Kar, Zonunfeli Ralte

June 2026

Intermediate

486 pages

15h 6m

English

Packt Publishing

Read now

Unlock full access

Preface
Free benefits with your book
Chapter 1: Foundations of LLM Evaluation: Core Concepts and Primitives
What evaluation means for LLM productsEvaluation is a decision system, not a scoreQuality is only one axisMetrics are not decisionsWhat evaluation is notRunning example setup: The production assistantWhy traditional ML evaluation breaks for generative and agentic systemsOpen-ended outputs break "single ground truth"Stochasticity breaks "one run = one answer"Prompt and context coupling breaks the model-centric worldviewAgents break static input-output assumptionsDefining "working" in production: quality, safety, cost, reliabilityThe four-axis definition of working (Q/S/C/R)Turning product goals into measurable signalsRisk tiers and severity mappingDefinition of "done" for the running assistant, plus rollback criteriaEvaluation primitives: the shared vocabularyEvaluation constructs: from primitives to decisionsEvaluators, metrics, actions, and thresholdsMetadata and versioningPrimitives and what to logBuilding test suites and datasetsSlice-based sets: Coverage that reflects realityGold sets: What "gold" means for generative tasksAdversarial sets: Intentional stress testsSynthetic sets: Scale with cautionEvaluator typesDeterministic validatorsReference-based metricsLLM-as-a-judgeHuman evaluation operationsRepeatability and statistical sanity under nondeterminismWhat repeatability means for stochastic systemsPractical variance controlsStatistical hygiene for product teamsEvaluator driftRegression detection under noiseFrom primitives to pipelines: Offline evaluation, online evaluation, regression gatesOffline evaluation pipelineOnline evaluation pipelineRegression gates: turning metrics into deployment decisionsLinking offline and online: closed-loop improvementBenchmarks vs.production evaluation: What transfers, what misleads, and whyEvaluation granularityTooling patterns: Harness design, versioning, dashboards, trace storageHarness design principlesVersioning as an evaluation requirementDashboards for decisions, not decorationTrace storage for debugging and evaluationTooling as the bridge from concepts to operationsRunning example: An end-to-end evaluation blueprint for a production assistantStep 1: Define the system tupleStep 2: Define representative casesStep 3: Organize cases into suites and slicesStep 4: Choose evaluators by failure modeStep 5: Run the offline evaluation pipelineStep 6: Apply gates and connect offline to onlineStep 7: Keep the blueprint lightweight at firstSummary
Chapter 2: Building Reliable Text-Only LLMs Through Training-Time Evaluation
Text-only system boundary, training stages, and training-time evaluation in practiceDefining the text-only system boundaryTraining stages require different evaluation lensesThe training-time evaluation loopData quality gates: Provenance, deduplication, toxicity, format consistency, instruction hygieneProvenance and deduplicationToxicity and safety content screeningFormat consistency and parser stabilityInstruction hygieneCoverage and representativeness across slicesCoverage is slice completeness, not more tokensLong-tail behavior and edge-case budgetsLeakage and contamination: Overlap, benchmark contamination, and memorization riskWhat counts as leakage in practiceBenchmark contamination is leakage with higher stakesDetecting overlap at scalePretraining objective validation: What training loss can and cannot tell youInstruction-tuning evaluation: Instruction-following, constraint adherence, and structured outputsTreat instruction following as a set of behaviorsConstraint adherence: evaluate rules as explicit checks where possibleStructured outputs as a first-class axisPreference tuning evaluation: Trade-offs, over-refusal, and mode collapseMake trade-offs explicit with a small drift scorecardOver-refusal is a slice problem, not a global problemMode collapse: "safe but unhelpful" repliesVerbosity driftGating preference tuning outcomesTask-specific fine-tuning: Accuracy, error taxonomies, and edge-case budgetsSafety alignment during training: Harmful completion rate and jailbreak robustnessHarmful completion rate and refusal outcomesJailbreak robustnessGeneralization and robustness: Out-of-distribution (OOD) sets, paraphrase stability, and adversarial suitesEfficiency signals that matter later: Tokens, lengths, and compute implicationsRelease readiness: Thresholds, model cards, dataset cards, and changelogsDecision framework: Selecting a stronger base model versus fine-tuningSummary
Chapter 3: Controlling Text-Only LLM Behavior at Inference Time
The inference pipeline and why it needs controlsInput validation and intake policiesPrompt governance as versioned product codeOutput contracts and the bounded reject–repair fallback loopDecoding policy and stability under nondeterminismRuntime evaluation with calibrated judges and human samplingTool calling correctness and evidence-bound claimsRetrieval-augmented generation controls and groundednessSafety controls as slice-based gatesPerformance and cost as first-class budgetsMonitoring, drift, and incident responseRollout patterns: Shadow, canary, and rollbackEnd-to-end inference control blueprint for the running assistantSummary
Chapter 4: Grounding and Reliability in Vision Language Models During Training
What makes a vision language model a different evaluation problem?Anatomy of a vision-language model as an evaluation targetDefining grounding: From intuition to measurable propertyWhat transfers from text-only evaluation and what breaksProduction tasks, user journeys, and the failure modes they exposeMapping VLM production tasks to evaluation requirementsA production-oriented failure taxonomy for VLMsBuilding evaluation data for visual groundingGold set design for VLMs: Beyond image–caption pairsSynthetic and programmatically generated evaluation dataAdversarial and stress-test probes for groundingSignals and metrics for training-time VLM evaluationGrounding fidelity metricsHallucination detection and quantificationRobustness and consistency signalsCross-modal alignment metricsPretraining objective validation: Contrastive signal health and in-training retrieval monitoringComposing a training-time evaluation dashboardEvaluator selection and calibrationHuman evaluation for visual grounding: Design and pitfallsDeterministic and programmatic evaluatorsModel-based graders for VLM evaluationFrom signals to training decisions: Thresholds, gates, and readinessSetting thresholds for grounding and hallucination metricsMultimodal safety evaluation as a hard training gateCheckpoint selection and training stage gatesEfficiency signals as a parallel deployment gateDiagnosing and responding to evaluation failuresTooling patterns for VLM training-time evaluationEvaluation harness architecture for VLMsVersioning, traceability, and reproducibilityDashboards, regression reports, and communication artifactsCase study: Training-Time evaluation for a production visual document understanding VLMSystem definition and task requirementsFailure taxonomy applied to document understandingEvaluation data, metrics, and evaluator choices in practiceThresholds, gate decisions, and a training retrospectiveSummary
Chapter 5: Evaluating Visual Grounding and Reliability at Inference Time
The VLM inference pipelineImage intake and input validationVisual context handling: What the model actually seesPrompt governance for multimodal inputsOutput contracts for grounded structured outputsDecoding policy and stability under visual nondeterminismRuntime grounding evaluation with calibrated graders and human samplingEvidence-bound visual claimsVisual grounding controls and groundedness at runtimeGrounding when retrieval is added to the VLM pipelineMultimodal safety controls as runtime slice-based gatesPerformance and cost as first-class budgets at production resolutionsMonitoring, visual drift, and incident responseRollout patterns for VLM systems: Shadow, canary, rollbackEnd-to-end inference control blueprint for the document-understanding VLMThe inference control system, station by stationA walkthrough incidentSummary
Chapter 6: Evaluating Multimodal Conversational LLMs Across Training and Inference
Architecture of multimodal conversational LLMsMultimodal LLM evaluation lifecyclePhase 1: Problem definition and KPI alignmentPhase 2: Multimodal data quality assessmentPhase 3: Pretraining objective validationPhase 4: Instruction and modality alignment evaluationPhase 5: Task-specific fine-tuning evaluationPhase 6: Generalization and robustness testingPhase 7: Inference stage safeguardsPhase 8: Runtime decoding stabilityPhase 9: Operational and cost evaluationPhase 10: Conversational outcome evaluationMultimodal data quality and coverage for conversational LLMsData quality in multimodal conversational LLMsData coverage in multimodal conversational LLMsData readiness evaluation for multimodal conversational trainingMajor KPIs for data quality, coverage, and business performancePretraining objective validation for speech, vision, and text alignmentInstruction and modality alignment tuningTask-specific fine tuning for multimodal conversationGeneralization and robustness across modalitiesInput validation and multimodal context handling at inferenceDecoding stability and cross-modality reasoning at runtimeTranscription drift and reasoning degradation detectionEnd-to-end latency, throughput, and cost evaluationConversational success, user friction, and fallback effectivenessLife cycle framework to the detailed evaluation dimensionsFrom KPI definition to practical measurementSummary
Chapter 7: Evaluating Routing and Reliability in Mixture of Experts LLMs
Architecture of mixture of expert LLMsData quality and coverage for expert trainingData quality in MoE trainingData coverage in MoE trainingData readiness evaluation for MoETools for data quality evaluationChallenges specific to MoEKey performance indicators for data quality and coveragePretraining objective validation and router learningInstruction and alignment tuning across expertsTask-specific fine tuning and expert specializationGeneralization, load balance, and expert collapse detectionInput validation and routing decisions at inferenceRouting stability and cold expert activationLatency variance, throughput, and cost evaluationOutput quality consistency across expertsToken dropping in MoE generationSummary
Chapter 8: Evaluating Reliability and Control in Computer-Using Agent Systems
Architecture of computer-using agent LLMs, and what are large action modelsComparing RPA, CUA systems, and LAMsRobotic process automationComputer-using agentsLarge action modelsFara-7B as a unified CUA-style architectureArchitecture of xLAM from SalesforcePossible implementation of a CUA pipelinePretraining objective validation for action groundingFara-7BxLAM-7b-rInstruction and alignment tuning for multi-step actionsFara-7BxLAM-7b-rTask-specific fine-tuning and step feasibilityFara-7BxLAM-7b-rTraining-time generalization and robustness in agent modelsFara-7BxLAM-7b-rInference-time evaluation of hybrid CUA architecturesInference-time generalization and robustness in agent environmentsInput, state validation, and action selection at inferenceStep success, tool failure recovery, and loop detectionLatency, throughput, and cost in long-horizon executionTask completion, safety violations, and long-horizon stabilityEvaluating tool- and function-calling SLMsSummary
Chapter 9: Evaluating Information Extraction and Document-Understanding LLMs
Named-entity recognition using LLMsEvaluating information extraction (NER) using LLMsTraining evaluation for span accuracy and label consistencyFine-tuning for improved span precision and label consistencyBoundary errors and entity confusion analysisRobustness to noisy and adversarial text at inferenceDownstream extraction accuracy and error propagationOptical character recognition using LLMsTraining time evaluation for text layout grounding and transcription accuracySensitivity to image quality and resolution at inferenceLatency and throughput in document processingDocument processing accuracy and automation successApplication of OCR and NER pipelinesSimulated OCR-NER model pipelineReal OCR-NER model pipelineSummary

Chapter 10: Evaluating Reasoning LLMs in Depth
The evolution of reasoning in LLMsDefining reasoning in LLMs and SLMsModern reasoning models and evaluation challengesCore architectural components of reasoning modelsTraining evaluation for logical consistency and chain correctnessLogical consistency during trainingChain-of-thought supervisionEvaluating intermediate reasoning stepsMeasuring reasoning depth and generalizationUnderstanding reasoning depthMeasuring generalization beyond memorizationStep-level coherence and nondeterminism at inferenceStep-level coherence in reasoning modelsWhy nondeterminism emerges in DeepSeek-R1Measuring step-level coherenceDecoding stability and error propagation in reasoning tasksDecoding strategies and their trade-offsGreedy decodingTemperature samplingBeam searchSelf-consistency decodingMoE-specific error propagation after decodingDecoding stability in enterprise systemsReliability of reasoning outputs in real use casesCalibrating LLM-as-a-judge frameworksSummary
Chapter 11: Evaluating Specialized LLM Systems
Structured query generation systems: Text2SQL and Text2CypherUnderstanding and evaluating Text2SQL SLMsTraining-time evaluation flow for non-native Text2SQL systemsTraining-time evaluation flow for native fine-tuned Text2SQL systemsInference-time evaluation for non-native Text2SQL systemsInference-time evaluation for native fine-tuned Text2SQL systemsUnderstanding and evaluating Text2Cypher SLMsUnderstanding the Gemma 2 Text2Cypher modelEvaluating Gemma 2 Text2Cypher systems across training and inferenceUnderstanding and evaluating Guardrail SLMsEvaluating ShieldGemma for runtime hate-speech policy enforcementTraining-time evaluationInference-time evaluationEvaluating ShieldGemma for multimodal image safety and visual governanceUnderstanding and evaluating embedding SLMsUnderstanding embeddingsEvaluating embedding SLMsEvaluating text embedding SLMsEvaluating image embedding SLMsEvaluating multimodal embedding SLMsSummary
Chapter 12: Unlock Access to the Code Bundle and the PDF Version
Unlock this book's free benefits in three easy steps
Index

Content preview from Practical LLM Evaluation for Production Systems

8 Evaluating Reliability and Control in Computer-Using Agent Systems

Computer-Using Agents (CUAs) represent a shift from traditional language-model applications that generate text to systems that can act within digital environments through tools, APIs, browsers, user interfaces, and multi-step workflows. In this chapter, CUA refers to a general category of LLM-based systems that can operate within digital environments by using tools, APIs, browsers, user interfaces, and multi-step workflows. These agents are capable of planning, executing, and adapting sequences of actions to achieve a goal, often operating over extended horizons with feedback from external systems.

Consider an enterprise order-management agent that is asked to cancel a shipment ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781807423896

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Practical LLM Evaluation for Production Systems

by Ammar Mohanna, Indrajit Kar, Zonunfeli Ralte

8

Evaluating Reliability and Control in Computer-Using Agent Systems

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.