book

AI Engineering

by Chip Huyen

December 2024

Intermediate to advanced

534 pages

15h 52m

English

O'Reilly Media, Inc.

Audio summary available

Read now

Unlock full access

Includes

Includes Quizzes

What This Book Is AboutWhat This Book Is NotWho This Book Is ForNavigating This BookConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
The Rise of AI EngineeringFrom Language Models to Large Language ModelsFrom Large Language Models to Foundation ModelsFrom Foundation Models to AI EngineeringFoundation Model Use CasesCodingImage and Video ProductionWritingEducationConversational BotsInformation AggregationData OrganizationWorkflow AutomationPlanning AI ApplicationsUse Case EvaluationSetting ExpectationsMilestone PlanningMaintenanceThe AI Engineering StackThree Layers of the AI StackAI Engineering Versus ML EngineeringAI Engineering Versus Full-Stack EngineeringSummary
Training DataMultilingual ModelsDomain-Specific ModelsModelingModel ArchitectureModel SizePost-TrainingSupervised FinetuningPreference FinetuningSamplingSampling FundamentalsSampling StrategiesTest Time ComputeStructured OutputsThe Probabilistic Nature of AISummary
Challenges of Evaluating Foundation ModelsUnderstanding Language Modeling MetricsEntropyCross EntropyBits-per-Character and Bits-per-BytePerplexityPerplexity Interpretation and Use CasesExact EvaluationFunctional CorrectnessSimilarity Measurements Against Reference DataIntroduction to EmbeddingAI as a JudgeWhy AI as a Judge?How to Use AI as a JudgeLimitations of AI as a JudgeWhat Models Can Act as Judges?Ranking Models with Comparative EvaluationChallenges of Comparative EvaluationThe Future of Comparative EvaluationSummary
Evaluation CriteriaDomain-Specific CapabilityGeneration CapabilityInstruction-Following CapabilityCost and LatencyModel SelectionModel Selection WorkflowModel Build Versus BuyNavigate Public BenchmarksDesign Your Evaluation PipelineStep 1. Evaluate All Components in a SystemStep 2. Create an Evaluation Guideline Step 3. Define Evaluation Methods and DataSummary
Introduction to PromptingIn-Context Learning: Zero-Shot and Few-ShotSystem Prompt and User PromptContext Length and Context EfficiencyPrompt Engineering Best PracticesWrite Clear and Explicit InstructionsProvide Sufficient ContextBreak Complex Tasks into Simpler SubtasksGive the Model Time to ThinkIterate on Your PromptsEvaluate Prompt Engineering ToolsOrganize and Version PromptsDefensive Prompt EngineeringProprietary Prompts and Reverse Prompt EngineeringJailbreaking and Prompt InjectionInformation ExtractionDefenses Against Prompt AttacksSummary
RAGRAG ArchitectureRetrieval AlgorithmsRetrieval OptimizationRAG Beyond TextsAgentsAgent OverviewToolsPlanningAgent Failure Modes and EvaluationMemorySummary
Finetuning OverviewWhen to FinetuneReasons to FinetuneReasons Not to FinetuneFinetuning and RAGMemory BottlenecksBackpropagation and Trainable ParametersMemory MathNumerical RepresentationsQuantizationFinetuning TechniquesParameter-Efficient FinetuningModel Merging and Multi-Task FinetuningFinetuning TacticsSummary
Data CurationData QualityData CoverageData QuantityData Acquisition and AnnotationData Augmentation and SynthesisWhy Data SynthesisTraditional Data Synthesis TechniquesAI-Powered Data SynthesisModel DistillationData ProcessingInspect DataDeduplicate DataClean and Filter DataFormat DataSummary
Understanding Inference OptimizationInference OverviewInference Performance MetricsAI AcceleratorsInference Optimization Model OptimizationInference Service OptimizationSummary

AI Engineering ArchitectureStep 1. Enhance ContextStep 2. Put in GuardrailsStep 3. Add Model Router and GatewayStep 4. Reduce Latency with CachesStep 5. Add Agent PatternsMonitoring and ObservabilityAI Pipeline OrchestrationUser FeedbackExtracting Conversational FeedbackFeedback DesignFeedback LimitationsSummary

Content preview from AI Engineering

About the Author

Chip Huyen is a writer and computer scientist specializing in machine learning (ML) systems. She has worked at NVIDIA, Snorkel AI, founded an AI infrastructure startup (later acquired), and taught ML systems at Stanford University.

This book draws on her experience helping major organizations and startups leverage AI for practical solutions. Her 2022 book, Designing Machine Learning Systems (O’Reilly), is an Amazon bestseller in AI and has been translated into over 10 languages.

She is also the author of four bestselling Vietnamese books, including the series Xach ba lo len va Di (Pack Your Bag and Go).

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Start your free trial

Publisher Resources

ISBN: 9781098166298Errata Page Supplemental Content

AI Engineering

by Chip Huyen

About the Author

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

You might also like

AI Engineering

AI Engineering

LLM Engineer's Handbook

Building LLM Powered Applications

Publisher Resources

About the Author

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,and much more.

You might also like

AI Engineering

AI Engineering

LLM Engineer's Handbook

Building LLM Powered Applications

Publisher Resources

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.