book

Generative AI on AWS

by Chris Fregly, Antje Barth, Shelbee Eigenbrode

November 2023

Intermediate to advanced

312 pages

8h 15m

English

O'Reilly Media, Inc.

Audiobook available

Read now

Unlock full access

Includes

Sandbox

Preface
Conventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgmentsChrisAntjeShelbee
1. Generative AI Use Cases, Fundamentals, and Project Life Cycle
Use Cases and TasksFoundation Models and Model HubsGenerative AI Project Life CycleGenerative AI on AWSWhy Generative AI on AWS?Building Generative AI Applications on AWSSummary
2. Prompt Engineering and In-Context Learning
Prompts and CompletionsTokensPrompt EngineeringPrompt StructureInstructionContextIn-Context Learning with Few-Shot InferenceZero-Shot InferenceOne-Shot InferenceFew-Shot InferenceIn-Context Learning Gone WrongIn-Context Learning Best PracticesPrompt-Engineering Best PracticesInference Configuration ParametersSummary
3. Large-Language Foundation Models
Large-Language Foundation ModelsTokenizersEmbedding VectorsTransformer ArchitectureInputs and Context WindowEmbedding LayerEncoderSelf-AttentionDecoderSoftmax OutputTypes of Transformer-Based Foundation ModelsPretraining DatasetsScaling Laws Compute-Optimal ModelsSummary
4. Memory and Compute Optimizations
Memory ChallengesData Types and Numerical PrecisionQuantizationfp16bfloat16fp8int8Optimizing the Self-Attention LayersFlashAttentionGrouped-Query AttentionDistributed ComputingDistributed Data ParallelFully Sharded Data ParallelPerformance Comparison of FSDP over DDPDistributed Computing on AWSFully Sharded Data Parallel with Amazon SageMakerAWS Neuron SDK and AWS TrainiumSummary
5. Fine-Tuning and Evaluation
Instruction Fine-TuningLlama 2-ChatFalcon-ChatFLAN-T5Instruction DatasetMultitask Instruction DatasetFLAN: Example Multitask Instruction DatasetPrompt TemplateConvert a Custom Dataset into an Instruction DatasetInstruction Fine-TuningAmazon SageMaker StudioAmazon SageMaker JumpStartAmazon SageMaker Estimator for Hugging FaceEvaluationEvaluation MetricsBenchmarks and DatasetsSummary
6. Parameter-Efficient Fine-Tuning
Full Fine-Tuning Versus PEFTLoRA and QLoRALoRA FundamentalsRankTarget Modules and LayersApplying LoRAMerging LoRA Adapter with Original ModelMaintaining Separate LoRA AdaptersFull-Fine Tuning Versus LoRA PerformanceQLoRAPrompt Tuning and Soft PromptsSummary
7. Fine-Tuning with Reinforcement Learning from Human Feedback
Human Alignment: Helpful, Honest, and HarmlessReinforcement Learning OverviewTrain a Custom Reward ModelCollect Training Dataset with Human-in-the-LoopSample Instructions for Human LabelersUsing Amazon SageMaker Ground Truth for Human AnnotationsPrepare Ranking Data to Train a Reward ModelTrain the Reward ModelExisting Reward Model: Toxicity Detector by MetaFine-Tune with Reinforcement Learning from Human FeedbackUsing the Reward Model with RLHFProximal Policy Optimization RL AlgorithmPerform RLHF Fine-Tuning with PPOMitigate Reward HackingUsing Parameter-Efficient Fine-Tuning with RLHFEvaluate RLHF Fine-Tuned ModelQualitative EvaluationQuantitative EvaluationLoad Evaluation ModelDefine Evaluation-Metric Aggregation FunctionCompare Evaluation Metrics Before and AfterSummary
8. Model Deployment Optimizations
Model Optimizations for InferencePruningPost-Training Quantization with GPTQDistillationLarge Model Inference ContainerAWS Inferentia: Purpose-Built Hardware for InferenceModel Update and Deployment StrategiesA/B TestingShadow DeploymentMetrics and MonitoringAutoscalingAutoscaling PoliciesDefine an Autoscaling PolicySummary
9. Context-Aware Reasoning Applications Using RAG and Agents
Large Language Model LimitationsHallucinationKnowledge CutoffRetrieval-Augmented GenerationExternal Sources of KnowledgeRAG WorkflowDocument Loading ChunkingDocument Retrieval and RerankingPrompt AugmentationRAG Orchestration and ImplementationDocument Loading and ChunkingEmbedding Vector Store and RetrievalRetrieval ChainsReranking with Maximum Marginal RelevanceAgentsReAct FrameworkProgram-Aided Language FrameworkGenerative AI ApplicationsFMOps: Operationalizing the Generative AI Project Life CycleExperimentation ConsiderationsDevelopment Considerations Production Deployment ConsiderationsSummary

10. Multimodal Foundation Models
Use CasesMultimodal Prompt Engineering Best PracticesImage Generation and EnhancementImage GenerationImage Editing and EnhancementInpainting, Outpainting, Depth-to-ImageInpaintingOutpaintingDepth-to-ImageImage Captioning and Visual Question AnsweringImage CaptioningContent ModerationVisual Question Answering Model EvaluationText-to-Image Generative TasksForward DiffusionNonverbal ReasoningDiffusion Architecture FundamentalsForward DiffusionReverse DiffusionU-Net Stable Diffusion 2 ArchitectureText EncoderU-Net and Diffusion ProcessText ConditioningCross-AttentionSchedulerImage DecoderStable Diffusion XL ArchitectureU-Net and Cross-AttentionRefinerConditioningSummary
11. Controlled Generation and Fine-Tuning with Stable Diffusion
ControlNetFine-TuningDreamBoothDreamBooth and PEFT-LoRATextual InversionHuman Alignment with Reinforcement Learning from Human FeedbackSummary
12. Amazon Bedrock: Managed Service for Generative AI
Bedrock Foundation ModelsAmazon Titan Foundation ModelsStable Diffusion Foundation Models from Stability AIBedrock Inference APIsLarge Language ModelsGenerate SQL CodeSummarize TextEmbeddingsFine-TuningAgentsMultimodal ModelsCreate Images from TextCreate Images from ImagesData Privacy and Network SecurityGovernance and MonitoringSummary
Index
About the Authors

Content preview from Generative AI on AWS

Chapter 3. Large-Language Foundation Models

In Chapter 2, you learned how to perform prompt engineering and leverage in-context learning using an existing foundation model. In this chapter, you will explore how a foundation model is trained, including the training objectives and datasets. While it’s not common to train your own foundation model from scratch, it is worth understanding how much time, effort, and complexity is required to perform this compute-intensive process.

Training a multibillion-parameter large-language model from scratch, called pretraining, requires millions of GPU compute hours, trillions of data tokens, and a lot of patience. In this chapter, you will learn about empirical scaling laws as described in the popular Chinchilla paper for model pretraining.¹

When training the BloombergGPT model, for example, researchers used the Chinchilla scaling laws as a starting point but still required a lot of trial and error, as explained in the BloombergGPT paper.² With a GPU compute budget of 1.3 million GPU hours, BloombergGPT was trained with a large distributed cluster of GPU instances using Amazon SageMaker.

Note

This chapter dives deep into pretraining generative foundation models, which may overwhelm some readers. It’s important to note that you do not need to fully understand this chapter to effectively build generative AI applications. You may find this chapter useful as a reference for some advanced concepts later in this book.

Large-Language Foundation ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Kubernetes for the Absolute Beginners - Hands-On

Publisher Resources

ISBN: 9781098159214Errata Page

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design