July 2018
Intermediate to advanced
310 pages
8h 21m
Chinese
Content preview from 深度学习入门 : 基于Python的理论与实现
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
第 6章 与学习相关的技巧
184
这个实验中，神经网络有5 层，每层有 100 个神经元，激活函数使用的
是
ReLU
。从图 6
-
15 的结果可知，
std =
0.01 时完全无法进行学习。这和刚
才观察到的激活值的分布一样，是因为正向传播中传递的值很小（集中在0
附近的数据）。因此，逆向传播时求到的梯度也很小，权重几乎不进行更新。
相反，当权重初始值为
Xavier
初始值和
He
初始值时，学习进行得很顺利。
并且，我们发现
He
初始值时的学习进度更快一些。
综上，在神经网络的学习中，权重初始值非常重要。很多时候权重初始
值的设定关系到神经网络的学习能否成功。权重初始值的重要性容易被忽视，
而任何事情的开始（初始值）总是关键的，因此在结束本节之际，再次强调
一下权重初始值的重要性。
6.3
Batch Normalization
在上一节，我们观察了各层的激活值分布，并从中了解到如果设定了合
适的权重初始值，则各层的激活值分布会有适当的广度，从而可以顺利地进
行学习。那么，为了使各层拥有适当的广度，“强制性”地调整激活值的分布
会怎样呢？实际上，
Batch Normalization
[11]
方法就是基于这个想法而产生的。
6.3.1
Batch Normalization 的算法
Batch Normalization
（下文简称
Batch Norm
）是 2015 年提出的方法。
Batch Norm
虽然是一个问世不久的新方法，但已经被很多研究人员和技术
人员广泛使用。实际上，看一下机器学习竞赛的结果，就会发现很多通过使 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.