July 2018
Intermediate to advanced
310 pages
8h 21m
Chinese
Content preview from 深度学习入门 : 基于Python的理论与实现
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
6.4 正则化
191
过了100 个
epoch
左右后，用训练数据测量到的识别精度几乎都为
100
%
。但是，对于测试数据，离100
%
的识别精度还有较大的差距。如此大
的识别精度差距，是只拟合了训练数据的结果。从图中可知，模型对训练时
没有使用的一般数据（测试数据）拟合得不是很好。
6.4.2
权值衰减
权值衰减是一直以来经常被使用的一种抑制过拟合的方法。该方法通过
在学习的过程中对大的权重进行惩罚，来抑制过拟合。很多过拟合原本就是
因为权重参数取值过大才发生的。
复习一下，神经网络的学习目的是减小损失函数的值。这时，例如为
损失函数加上权重的平方范数（
L
2 范数）。这样一来，就可以抑制权重变大。
用符号表示的话，如果将权重记为
W
，
L
2 范数的权值衰减就是
，然
后将这个
加到损失函数上。这里，
λ
是控制正则化强度的超参数。
λ
设置得越大，对大的权重施加的惩罚就越重。此外，
开头的 是用于
将
的求导结果变成
λ
W
的调整用常量。
对于所有权重，权值衰减方法都会为损失函数加上
。因此，在求权
重梯度的计算中，要为之前的误差反向传播法的结果加上正则化项的导数
λ
W
。
L2范数相当于各个元素的平方和。用数学式表示的话，假设有权重
W
= (w
1
,w
2
,
...
,w
n
)
，则L2范数可用
计算
出来。除了L2范数，还有L1范数、L∞范数等。L1范数是各个元
素的绝对值之和，相当于
|w
1
| + |w
2
| +
...
+ |w
n
|
。L∞范数也称为
Max范数，相当于各个元素的绝对值中最大的那一个。L2范数、L1
范数、L∞范数都可以用作正则化项，它们各有各的特点，不过这里 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.