May 2025
Intermediate to advanced
382 pages
10h 33m
Chinese
Content preview from 图解大模型 : 生成式AI 原理与实战
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
110
｜
第
4
章
4.6.2
使用
ChatGPT
进行分类
尽管在本书中我们主要关注开源模型，但专有模型（特别是
ChatGPT
）是语言人工智能领
域另一个不可或缺的部分。
虽然原始
ChatGPT
模型（
GPT-3.5
）的底层架构并未公开，但从其名称我们可以推测，它
是基于我们在
GPT
模型中所见的仅解码器架构的。
幸好，
OpenAI
分享了关于训练过程的大体情况，其中涉及一个重要组件，即偏好调优
（
preference tuning
）。如图
4-22
所示，
OpenAI
首先手动创建了输入提示词（指令数据）的
期望输出，并使用这些数据创建了模型的第一个版本。
生成模型
目标: 基于提示词
生成输出
...
The answer
is 2.
What is
1 + 1?
样例提示词
人类标注员创建
期望输出
指令微调
指令数据收集
图 4-22：由指令（提示词）和输出组成的手动标注数据被用于执行微调（指令微调）
OpenAI
使用上述过程得到的模型生成多个输出，并手动对这些输出从最好到最差进行排序。
如图
4-23
所示，这种排序展示了对某些输出的偏好（偏好数据），并被用于创建最终的模
型，即
ChatGPT
。
生成模型
目标：基于提示词
生成期望输出
> >
C
Explain
LLMs
Large language models
B
I am not familiar with...
A
C B A
An abbreviation for
the Master of Laws.
使用指令微调模型
生成输出
偏好调优
创建偏好数据
人类标注员对
输出进行排序
图 4-23：手动排序的偏好数据被用于生成最终的模型 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.