September 2022
Intermediate to advanced
549 pages
11h 27m
Chinese
Content preview from 基于AWS的数据科学实践
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
训练第一个模型
｜
249
图 7-15：训练过程中，BERT 在输入序列对上并行运行掩码和 NSP
欲了解更多关于
BERT
的细节，请查阅发表于
2018
年的论文：《
BERT: Pre-training
of Deep Bidirectional Transformers for Language Understanding
》（
BERT
：语言理解
的深度双向变换器预训练）（
https://oreil.ly/LP4yX
）。
大多数情况下，我们不需要从头训练
BERT
。神经网络被设计用于在新数据到达系
统时重用和持续训练。由于
BERT
已经使用成百上千万来自维基百科（
Wikipedia
）
和谷歌图书语料库（
Google Books Corpus
）的公有文档进行预训练，其学习的词汇
量和表示可以应用在众多不同领域的多数
NLP
和
NLU
任务中。
尽管从头训练
BERT
需要许多数据和计算，然而这种情况下，
BERT
可以使用高度
定制化的词库学习自定义数据集的表示。诸如
Amazon
和
LinkedIn
这样的公司都拥
有从头预训练的内部
BERT
版本，用于学习其领域特定的语言表示。以
LinkedIn
的
BERT
变体为例，其学习了特定于职业名称、简历、公司和商业新闻的语言模型。
7.7
微调预先训练的
BERT
模型
ELMo
、
GPT/GPT-2
和
BERT
内建了经过训练的特定模型，一般称为“预训练模型”。
这些模型由来源于众多不同领域的数以百万计的文档预训练，其不仅在预测缺失词
时表现突出，而且擅长于学习词义、句子结构和句子相关性。它们生成有意义的、 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.