October 2020
Intermediate to advanced
424 pages
11h 39m
Chinese
Content preview from 深度学习进阶：自然语言处理
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
第8 章 Attention
356
8.4
关于Attention 的其他话题
到目前为止，我们研究了
Attention
（正确地说，是带
Attention
的
seq2seq
），本节我们介绍几个之前未涉及的话题。
8.4.1
双向RNN
这里我们关注
seq2seq
的编码器。首先复习一下，上一节之前的编码器
如图 8
-
29 所示。
图8-29 基于LSTM 层输出
hs
hs
LSTM LSTM LSTM LSTM LSTM
Embedding Embedding Embedding Embedding Embedding
吾輩 は猫である
如图 8
-
29 所示，
LSTM
中各个时刻的隐藏状态向量被整合为
hs
。这里，
编码器输出的
hs
的各行中含有许多对应单词的成分。
需要注意的是，我们是从左向右阅读句子的。因此，在图 8
-
29 中，单
词“猫”的对应向量编码了“吾輩”“は”“猫”这 3 个单词的信息。如果考
8.4 关于Attention 的其他话题
357
虑整体的平衡性，我们希望向量能更均衡地包含单词“猫”周围的信息。
在这次的翻译问题中，我们获得了所有的时序数据（需要翻译的
文本）。因此，我们既可以从左向右读取文本，也可以从右向左
读取文本。
为此，可以让
LSTM
从两个方向进行处理，这就是名为双向 LSTM
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.