July 2018
Intermediate to advanced
310 pages
8h 21m
Chinese
Content preview from 深度学习入门 : 基于Python的理论与实现
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
6.1 参数的更新
175
上面我们介绍了
SGD
、
Momentum
、
AdaGrad
、
Adam
这 4 种方法，那
么用哪种方法好呢？非常遗憾，（目前）并不存在能在所有问题中都表现良好
的方法。这4种方法各有各的特点，都有各自擅长解决的问题和不擅长解决
的问题。
很多研究中至今仍在使用
SGD
。
Momentum
和
AdaGrad
也是值得一试
的方法。最近，很多研究人员和技术人员都喜欢用
Adam
。本书将主要使用
SGD
或者
Adam
，读者可以根据自己的喜好多多尝试。
6.1.8
基于MNIST数据集的更新方法的比较
我们以手写数字识别为例，比较前面介绍的
SGD
、
Momentum
、
AdaGrad
、
Adam
这 4 种方法，并确认不同的方法在学习进展上有多大程度
的差异。先来看一下结果，如图6
-
9 所示（源代码在
ch06/optimizer_compare_
mnist.py
中 ）。
图6-9 基于 MNIST 数据集的 4 种更新方法的比较：横轴表示学习的迭代次数（iteration），
纵轴表示损失函数的值（loss）
1.0
0.8
0.6
0.4
0.2
0.0
0 500 1000
iterations
1500 2000
loss
Adam
SGD
AdaGrad
Momentum
这个实验以一个5 层神经网络为对象，其中每层有100 个神经元。激活
函数使用的是
ReLU
。
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.