book

Prompt Engineering for LLMs

by John Berryman, Albert Ziegler

November 2024

Intermediate to advanced

282 pages

8h 1m

English

O'Reilly Media, Inc.

Book available

Read now

Unlock full access

Includes

Includes Quizzes

Who Is This Book For?What You Will LearnConventions Used in This BookO’Reilly Online LearningHow to Contact UsAcknowledgmentsFrom JohnFrom Albert
LLMs Are MagicLanguage Models: How Did We Get Here?Early Language ModelsGPT Enters the ScenePrompt EngineeringConclusion
What Are LLMs?Completing a DocumentHuman Thought Versus LLM ProcessingHallucinationsHow LLMs See the WorldDifference 1: LLMs Use Deterministic TokenizersDifference 2: LLMs Can’t Slow Down and Examine LettersDifference 3: LLMs See Text DifferentlyCounting TokensOne Token at a TimeAuto-Regressive ModelsPatterns and RepetitionsTemperature and ProbabilitiesThe Transformer ArchitectureConclusion
Reinforcement Learning from Human FeedbackThe Process of Building an RLHF ModelKeeping LLMs HonestAvoiding Idiosyncratic BehaviorRLHF Packs a Lot of Bang for the BuckBeware of the Alignment TaxMoving from Instruct to ChatInstruct ModelsChat ModelsThe Changing APIChat Completion APIComparing Chat with CompletionMoving Beyond Chat to ToolsPrompt Engineering as PlaywritingConclusion
The Anatomy of the LoopThe User’s ProblemConverting the User’s Problem to the Model DomainUsing the LLM to Complete the PromptTransforming Back to User DomainZooming In to the Feedforward PassBuilding the Basic Feedforward PassExploring the Complexity of the LoopEvaluating LLM Application QualityOffline EvaluationOnline EvaluationConclusion
Sources of ContentStatic ContentClarifying Your QuestionFew-Shot PromptingDynamic ContentFinding Dynamic ContextRetrieval-Augmented GenerationSummarizationConclusion
Anatomy of the Ideal PromptWhat Kind of Document?The Advice ConversationThe Analytic ReportThe Structured DocumentFormatting SnippetsMore on InertnessFormatting Few-Shot ExamplesElastic SnippetsRelationships Among Prompt ElementsPositionImportanceDependencyPutting It All TogetherConclusion
Anatomy of the Ideal CompletionThe PreambleRecognizable Start and EndPostscriptBeyond the Text: LogprobsHow Good Is the Completion?LLMs for ClassificationCritical Points in the PromptChoosing the ModelConclusion

Tool UsageLLMs Trained for Tool UsageGuidelines for Tool DefinitionsReasoningChain of ThoughtReAct: Iterative Reasoning and ActionBeyond ReActContext for Task-Based InteractionsSources for ContextSelecting and Organizing ContextBuilding a Conversational AgentManaging ConversationsUser ExperienceConclusion
Would a Conversational Agent Suffice?Basic LLM WorkflowsTasksAssembling the WorkflowExample Workflow: Shopify Plug-in MarketingAdvanced LLM WorkflowsAllowing an LLM Agent to Drive the WorkflowStateful Task AgentsRoles and DelegationConclusion
What Are We Even Testing?Offline EvaluationExample SuitesFinding SamplesEvaluating SolutionsSOMA AssessmentOnline EvaluationA/B TestingMetricsConclusion
MultimodalityUser Experience and User InterfaceIntelligenceConclusion

Content preview from Prompt Engineering for LLMs

Chapter 8. Conversational Agency

In Chapter 3, we covered the departure from text completion models to chat models. A chat model by itself is aware of only the information covered in training and whatever information the user has just told it. The chat model is unable to reach out into the world and learn about information that was unavailable during training, and it’s unable to interact with the world and take external actions on behalf of the user.

The LLM community is making great headway in overcoming these limitations through conversational agency. Agency is the ability of an entity to complete tasks and achieve goals in a self-directed and autonomous manner. The conversational agents that we discuss in this chapter provide an experience similar to chat—a back-and-forth dialogue between a user and an assistant—but add in the ability for the assistant to reach out to the real world, learn new information, and interact with real-world assets.

In this chapter, we’ll introduce several state-of-the-art approaches to building an LLM-based conversational agent. We’ll explore how models can use tools to reach out into the external world, how they can be conditioned to better reason through their problem space, and how we can gather the best context to facilitate long or complex interactions. By the end of this chapter, you’ll be able to build your own conversational agent that’s capable of going out into the world and performing guided tasks on your behalf.