July 2018
Intermediate to advanced
310 pages
8h 21m
Chinese
Content preview from 深度学习入门 : 基于Python的理论与实现
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
第 6章 与学习相关的技巧
174
Adam会设置 3个超参数。一个是学习率（论文中以
α
出现），另外两
个是一次momentum 系数
β
1
和二次momentum 系数
β
2
。根据论文，
标准的设定值是
β
1
为0.9，
β
2
为0.999。设置了这些值后，大多数情
况下都能顺利运行。
6.1.7
使用哪种更新方法呢
到目前为止，我们已经学习了4 种更新参数的方法。这里我们来比较一
下这4 种方法（源代码在
ch06/optimizer_compare_naive.py
中 ）。
如图6
-
8 所示，根据使用的方法不同，参数更新的路径也不同。只看这
个图的话，
AdaGrad
似乎是最好的，不过也要注意，结果会根据要解决的问
题而变。并且，很显然，超参数（学习率等）的设定值不同，结果也会发生变化。
图6-8 最优化方法的比较：SGD、Momentum、AdaGrad、Adam
x
−10 −5 0 5 10
x
−10 −5 0 5 10
x
−10 −5 0 5
10
−10
−5
0
5
10
y
−10
−5
0
5
10
y
SGD
Momentum
AdaGrad Adam
−10
−5
0
5
10
y
x
−10 −5 0 5
10
−10
−5
0
5
10
y
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.