September 2022
Intermediate to advanced
549 pages
11h 27m
Chinese
Content preview from 基于AWS的数据科学实践
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
训练第一个模型
｜
245
7.5 BERT
转换器架构
在
BERT
的核心，转换器架构使用一种注意力机制以便其在遍历语料库时“注意”
特定且有趣的词和短语。具体来说，
BERT
转换器使用“自注意力”将数据中的每
一个符号与输入序列中的所有其他符号进行关联。此外，
BERT
使用“多头注意力
（
multiheaded attention
）”以处理词汇的歧义，也称为“多义性”（
polysemy
，希
腊语中
poly
即“多”，
sema
即“意义”）。注意力的一个示例如图
7-10
所示，
it
一词与
movie
一词高度关联，并与
funny
和
great
两词相关，然而相关性较
movie
一
词稍弱。
对文本
“This move is funny, it is great”
的注意力
图 7-10：“自注意力”机制将数据中每一符号与输入序列中其他所有符号进行关联
没有双向注意力的算法可能会对以下两个句子中的
bank
一词创建相同的嵌入：“
A
thief stole money from the bank vault
（一个窃贼从银行保险箱中窃取钱财）”和“
Later,
he was arrested while fishing on a river bank
（稍晚，他在河边钓鱼时被逮捕）”。
请注意，
bank
一词在每句话中有不同含义。对于人类来说，我们很容易由人生经
验来区分，这是一种天然的“预训练”。但对于缺乏类似预训练的机器来说，这并
不容易。
BERT
通过从特定序列的上下文中，对每个符号学习不同的向量，来区分
这两个词（符号）。习得的符号向量称为“输入符号向量表示（ ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.