July 2018
Intermediate to advanced
310 pages
8h 21m
Chinese
Content preview from 深度学习入门 : 基于Python的理论与实现
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
6.2 权重的初始值
181
图6-13 使用Xavier 初始值作为权重初始值时的各层激活值的分布
6000
5000
4000
3000
2000
1000
1-layer 2-layer 3-layer 4-layer 5-layer
0
0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0
图 6-13 的分布中，后面的层的分布呈稍微歪斜的形状。如果用
tanh
函数（双曲线函数）代替
sigmoid
函数，这个稍微歪斜的问题就能得
到改善。实际上，使用
tanh
函数后，会呈漂亮的吊钟型分布。
tanh
函数和
sigmoid
函数同是 S型曲线函数，但
tanh
函数是关于原点 (0, 0)
对称的S型曲线，而
sigmoid
函数是关于(x, y)
=
(0, 0.5)对称的S型曲
线。众所周知，用作激活函数的函数最好具有关于原点对称的性质。
6.2.3
ReLU的权重初始值
Xavier
初始值是以激活函数是线性函数为前提而推导出来的。因为
sigmoid
函数和
tanh
函数左右对称，且中央附近可以视作线性函数，所以适
合使用
Xavier
初始值。但当激活函数使用
ReLU
时，一般推荐使用
ReLU
专
用的初始值，也就是
Kaiming He
等人推荐的初始值，也称为“
He
初始值”
[10]
。
当前一层的节点数为n 时，
He
初始值使用标准差为
的高斯分布。当
Xavier ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.