book

How Large Language Models Work

Name: How Large Language Models Work
Author: Drew Farris
ISBN: 9781633437081

by Drew Farris

June 2025

Intermediate to advanced

200 pages

6h 20m

English

Manning Publications

Read now

Unlock full access

How Large Language Models Work
copyright
contents
preface
acknowledgments
about this book
about the authors
about the cover illustration
1 Big picture: What are LLMs?
1.1 Generative AI in context1.2 What you will learn1.3 Introducing how LLMs work1.4 What is intelligence, anyway?1.5 How humans and machines represent language differently1.6 Generative Pretrained Transformers and friends1.7 Why LLMs perform so well1.8 LLMs in action: The good, bad, and scary
2 Tokenizers: How large language models see the world
2.1 Tokens as numeric representations2.2 Language models see only tokens2.2.1 The tokenization process2.2.2 Controlling vocabulary size in tokenization2.2.3 Tokenization in detail2.2.4 The risks of tokenization2.3 Tokenization and LLM capabilities2.3.1 LLMs are bad at word games2.3.2 LLMs are challenged by mathematics2.3.3 LLMs and language equity2.4 Check your understanding2.5 Tokenization in context

3 Transformers: How inputs become outputs
3.1 The transformer model3.1.1 Layers of the transformer model3.2 Exploring the transformer architecture in detail3.2.1 Embedding layers3.2.2 Transformer layers3.2.3 Unembedding layers3.3 The tradeoff between creativity and topical responses3.4 Transformers in context
4 How LLMs learn
4.1 Gradient descent4.1.1 What is a loss function?4.1.2 What is gradient descent?4.2 LLMs learn to mimic human text4.2.1 LLM reward functions4.3 LLMs and novel tasks4.3.1 Failing to identify the correct task4.3.2 LLMs cannot plan4.4 If LLMs cannot extrapolate well, can I use them?4.5 Is bigger better?
5 How do we constrain the behavior of LLMs?
5.1 Why do we want to constrain behavior?5.1.1 Base models are not very usable5.1.2 Not all model outputs are desirable5.1.3 Some cases require specific formatting5.2 Fine-tuning: The primary method of changing behavior5.2.1 Supervised fine-tuning5.2.2 Reinforcement learning from human feedback5.2.3 Fine-tuning: The big picture5.3 The mechanics of RLHF5.3.1 Beginning with a naive RLHF5.3.2 The quality reward model5.3.3 The similar-but-different RLHF objective5.4 Other factors in customizing LLM behavior5.4.1 Altering training data5.4.2 Altering base model training5.4.3 Altering the outputs5.5 Integrating LLMs into larger workflows5.5.1 Customizing LLMs with retrieval augmented generation5.5.2 General-purpose LLM programming
6 Beyond natural language processing
6.1 LLMs for software development6.1.1 Improving LLMs to work with code6.1.2 Validating code generated by LLMs6.1.3 Improving code via formatting6.2 LLMs for formal mathematics6.2.1 Sanitized input6.2.2 Helping LLMs understand numbers6.2.3 Math LLMs also use tools6.3 Transformers and computer vision6.3.1 Converting images to patches and back6.3.2 Multimodal models using images and text6.3.3 Applicability of prior lessons
7 Misconceptions, limits, and eminent abilities of LLMs
7.1 Human rate of learning vs. LLMs7.1.1 The limitations on self-improvement7.1.2 Few-shot learning7.2 Efficiency of work: A 10-watt human brain vs. a 2000-watt computer7.2.1 Power7.2.2 Latency, scalability, and availability7.2.3 Refinement7.3 Language models are not models of the world7.4 Computational limits: Hard problems are still hard7.4.1 Using fuzzy algorithms for fuzzy problems7.4.2 When close enough is good enough for hard problems
8 Designing solutions with large language models
8.1 Just make a chatbot?8.2 Automation bias8.2.1 Changing the process8.2.2 When things are too risky for autonomous LLMs8.3 Using more than LLMs to reduce risk8.3.1 Combining LLM embeddings with other tools8.3.2 Designing a solution that uses embeddings8.4 Technology presentation matters8.4.1 How can you be transparent?8.4.2 Aligning incentives with users8.4.3 Incorporating feedback cycles
9 Ethics of building and using LLMs
9.1 Why did we build LLMs at all?9.1.1 The pros and cons of LLMs doing everything9.1.2 Do we want to automate all human work?9.2 Do LLMs pose an existential risk?9.2.1 Self-improvement and the iterative S-curve9.2.2 The alignment problem9.3 The ethics of data sourcing and reuse9.3.1 What is fair use?9.3.2 The challenges associated with compensating content creators9.3.3 The limitations of public domain data9.4 Ethical concerns with LLM outputs9.4.1 Licensing implications for LLM output9.4.2 Do LLM outputs poison the well?9.5 Other explorations in LLM ethics
References

Content preview from How Large Language Models Work

5 How do we constrain the behavior of LLMs?

This chapter covers

Constraining LLM behavior to make them more useful
The four areas where we can constrain LLM behavior
How fine-tuning allows us to update LLMs
How reinforcement learning can change the output of LLMs
Modifying the inputs of an LLM using retrieval augmented generation

It may seem counterintuitive that you can make a model more useful by controlling the output the model is allowed to produce, but it is almost always necessary when working with LLMs. This control is necessitated by the fact that when presented with an arbitrary text prompt, an LLM will attempt to generate what it believes to be an appropriate response, regardless of its intended use. Consider a chatbot helping ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781633437081Publisher Support Publisher Website

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

How Large Language Models Work

by Drew Farris