July 2018
Intermediate to advanced
310 pages
8h 21m
Chinese
Content preview from 深度学习入门 : 基于Python的理论与实现
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
第 6章 与学习相关的技巧
172
这里需要注意的是，最后一行加上了微小值
1e-7
。这是为了防止当
self.h[key]
中有0 时，将 0 用作除数的情况。在很多深度学习的框架中，这
个微小值也可以设定为参数，但这里我们用的是
1e-7
这个固定值。
现在，让我们试着使用
AdaGrad
解决式（6.2）的最优化问题，结果如图 6
-
6
所示。
图6
-
6 基于AdaGrad 的最优化的更新路径
10
10
50−5−10
5
0
−5
−10
y
x
AdaGrad
由图6
-
6 的结果可知，函数的取值高效地向着最小值移动。由于y 轴方
向上的梯度较大，因此刚开始变动较大，但是后面会根据这个较大的变动按
比例进行调整，减小更新的步伐。因此，y轴方向上的更新程度被减弱，“之”
字形的变动程度有所衰减。
6.1.6
Adam
Momentum
参照小球在碗中滚动的物理规则进行移动，
AdaGrad
为参
数的每个元素适当地调整更新步伐。如果将这两个方法融合在一起会怎么样
6.1 参数的更新
173
呢？这就是
Adam
[8]
方法的基本思路
A
。
Adam
是2015年提出的新方法。它的理论有些复杂，直观地讲，就是融
合了
Momentum
和
AdaGrad
的方法。通过组合前面两个方法的优点，有望
实现参数空间的高效搜索。此外，进行超参数的“偏置校正”也是
Adam ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.