July 2018
Intermediate to advanced
310 pages
8h 21m
Chinese
Content preview from 深度学习入门 : 基于Python的理论与实现
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
第 6章 与学习相关的技巧
168
图6-3 基于SGD 的最优化的更新路径：呈“之”字形朝最小值 (0, 0) 移动，效率低
10
10
50−5−10
5
0
−5
−10
y
x
SGD
6.1.4
Momentum
Momentum
是
“动量”的意思，和物理有关。用数学式表示
Momentum
方
法，如下所示。
（6.3）
（6.4）
和前面的
SGD
一样，
W
表示要更新的权重参数，
表示损失函数关
于
W
的梯度，
η
表示学习率。这里新出现了一个变量
v
，对应物理上的速度。
式（6.3）表示了物体在梯度方向上受力，在这个力的作用下，物体的速度增
加这一物理法则。如图6
-
4 所示，
Momentum
方法给人的感觉就像是小球在
地面上滚动。
6.1 参数的更新
169
图6-4 Momentum：小球在斜面上滚动
式（6.3）中 有
α
v
这一项。在物体不受任何力时，该项承担使物体逐渐减
速的任务（
α
设定为0.9 之类的值），对应物理上的地面摩擦或空气阻力。下
面是
Momentum
的代码实现（源代码在
common/optimizer.py
中 ）。
class Momentum:
def __init__(self, lr=0.01, momentum=0.9):
self.lr = lr
self.momentum = momentum
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.