April 2020
Intermediate to advanced
816 pages
18h 32m
Chinese
Content preview from 精通機器學習
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
508
|
第十六章：用 RNN 與 Attention 處理自然語言
用
RNN
來建構能夠執行神經機器翻譯
（NMT）
的
Encoder–Decoder
結構
。
為此
，
我們將
使用
TensorFlow Addons
專案的
seq2seq API。
在本章的第二部分
，
我們將探討
專注機制
（attention mechanism）。
從名稱可以知道
，
這些
神經網路元件的工作是學習從輸入中選出模型的其他部分在各個時步應該專注的部分
。
我
們將先瞭解如何使用專注機制來提升
RNN Encoder–Decoder
結構的效果
，
接著完全捨棄
RNN，
研究一種非常成功的專注結構
，
稱為
Transformer。
最後
，
我們將回顧
NLP
在
2018
年到
2019
年的一些重大進展
，
包括非常強大的語言模型
，
例如
GPT-2
與
BERT，
它們的基
礎都是
Transformer。
我們先從一種簡單且有趣的模型開始看起
，
這種模型可以寫出類似莎士比亞文筆的文章
（
有點啦
！ ）。
用
Character RNN
產生莎翁文章
Andrej Karpathy
在一篇
2015
年發表的著名部落格文章
（https://homl.info/charrnn）「 The
Unreasonable Effectiveness of Recurrent Neural Networks」
中介紹如何訓練
RNN
來預測句子
的下一個字元
。
後來這種
Char-RNN
可用來產生新文章
，
每次一個字元
。
以下的短文是用
莎翁的所有文章來訓練的
Char-RNN
模型所產生的文字
：
PANDARUS:
Alas, I ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.