April 2020
Intermediate to advanced
816 pages
18h 32m
Chinese
Content preview from 精通機器學習
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
梯度消失 / 爆炸問題
|
329
公式
11-2 ELU
觸發函數
ELU
觸發函數看起來很像
ReLU，
但有一些重大的差異
：
•
它在
z < 0
時是負值
，
這可讓單位的平均輸出接近
0，
並且有助於緩解梯度消失問題
。
超參數
α
定義的是當
z
是大負數時
，ELU
接近哪個值
。
它通常被設為
1，
但是你可以
像任何其他超參數一樣調整它
。
•
當
z < 0
時
，
它的梯度不是零
，
可避免神經元死亡問題
。
•
α
等於
1
時
，
函數的任何地方都是平滑的
，
包括
z = 0
附近
，
這有助於提高梯度下降速
度
，
因為它在
z = 0
左右的彈跳不會太大
。
ELU
觸發函數的主要缺點是它的計算速度比
ReLU
函數及其變體慢
（
因為使用了指數函
數
）。
它在訓練時有更快的收斂速度
，
這可以彌補計算速度的不足
，
但是在測試時
，ELU
網路比
ReLU
慢
。
接著
，Günter Klambauer
等人在
2017
年發表的論文
（https://homl.info/selu）
7
中介紹了
Scaled ELU（SELU）
觸發函數
，
顧名思義
，
它是
ELU
觸發函數的尺度縮放版
（scaled
variant）。
作者展示
，
當你建立一個只由一疊稠密層組成
，
而且所有隱藏層都使用
SELU
觸
發函數的神經網路時
，
網路會
自動正規化
：
在訓練期間
，
各層的輸出都傾向保持均值為
0
且標準差為
1，
可解決梯度消失
/
爆炸問題
。
因此
，
使用這類的神經網路時
（
尤其是深
的
）， SELU
觸發函數的表現通常明顯優於其他觸發函數
。
但是自動正規化需要一些條件才
會發生
（
數學證明見論文
）：
•
輸入特徵必須標準化
（
均值
0，
且標準差
1）。
•
每個隱藏層的權重必須執行 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.