July 2018
Beginner to intermediate
432 pages
5h 58m
Japanese
Content preview from ゼロから作るDeep Learning ❷ ―自然言語処理編
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
付録CGRU
6章では「ゲート付きRNN」としてLSTMを詳しく説明しました。LSTMは大変良いレイヤですが、パラメータが多く、計算には時間がかかります。そこで最近では、LSTMに代わる「ゲート付きRNN」が数多く提案されています。ここでは、GRU（Gated Recurrent Unit）[42]と呼ばれる有名で実績のあるゲート付きRNNを紹介します。GRUは、LSTMのゲートを使用するというコンセプトはそのままで、パラメータを削減し、計算時間を短縮します。それでは、GRUの中身を見ていきましょう。
C.1 GRUのインタフェース
LSTMの重要な点は、ゲートを使用することです。それによって、学習時における勾配はスムーズに流れるため、勾配消失を緩和できます。この思想をGRUも受け継いでいます。ただし、LSTMとGRUはいくつか相違点が見られます。大きな違いは、図C-1のように、レイヤのインタフェースに見られます。
図C-1 LSTMとGRUの比較
図C-1に示すように、LSTMが隠れ状態と記憶セルの2つのラインを使用するのに対して、GRUは隠れ状態だけを使用します。ちなみに、これは5章で扱った「シンプルなRNN」と同じインタフェースです。
![[注記]](/api/v2/epubs/urn:orm:book:9784873118369/files/images/note.png) |
|
LSTMの記憶セルは、他のレイヤに見せることのない、プライベートな記憶部です。LSTMは記憶セルに必要な情報を記録し、記憶セルの情報を元に隠れ状態を求めました。それに対して、GRUは記憶セルのような追加の記憶部を必要としません。 ... |
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.