July 2018
Intermediate to advanced
310 pages
8h 21m
Chinese
Content preview from 深度学习入门 : 基于Python的理论与实现
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
第 6章 与学习相关的技巧
164
我们将指出
SGD
的缺点，并介绍
SGD
以外的其他最优化方法。
6.1.1
探险家的故事
进入正题前，我们先打一个比方，来说明关于最优化我们所处的状况。
有一个性情古怪的探险家。他在广袤的干旱地带旅行，坚持寻找幽
深的山谷。他的目标是要到达最深的谷底（他称之为“至深之地”）。这
也是他旅行的目的。并且，他给自己制定了两个严格的“规定”：一个
是不看地图；另一个是把眼睛蒙上。因此，他并不知道最深的谷底在这
个广袤的大地的何处，而且什么也看不见。在这么严苛的条件下，这位
探险家如何前往“至深之地”呢？他要如何迈步，才能迅速找到“至深
之地”呢？
寻找最优参数时，我们所处的状况和这位探险家一样，是一个漆黑的世
界。我们必须在没有地图、不能睁眼的情况下，在广袤、复杂的地形中寻找
“至深之地”。大家可以想象这是一个多么难的问题。
在这么困难的状况下，地面的坡度显得尤为重要。探险家虽然看不到周
围的情况，但是能够知道当前所在位置的坡度（通过脚底感受地面的倾斜状况）。
于是，朝着当前所在位置的坡度最大的方向前进，就是
SGD
的策略。勇敢
的探险家心里可能想着只要重复这一策略，总有一天可以到达“至深之地”。
6.1.2
SGD
让大家感受了最优化问题的难度之后，我们再来复习一下
SGD
。用数
学式可以将
SGD
写成如下的式（6.1）。
（6.1）
这里把需要更新的权重参数记为
W
，把损失函数关于
W
的梯度记为
。
η
表示学习率，实际上会取0.01 或 0.001 这些事先决定好的值。式子中的←
6.1 参数的更新 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.