October 2021
Intermediate to advanced
289 pages
8h 31m
Chinese
Content preview from 数据科学中的实用统计学(第2版)
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
228
｜
第
6
章
图 6-10：XGBoost 默认模型与惩罚模型的误差率比较
岭回归与
Lasso
这种添加一个对模型复杂度的惩罚项来避免过拟合的方法，可以追溯到
20
世纪
70
年
代。最小二乘回归最小化的是残差平方和（
RSS
），参见
4.1.3
节。
岭回归
最小化残差
平方和与一个惩罚项的和，这个惩罚项是关于系数数量和大小的一个函数：
22 2
01 1
1
ˆˆ ˆˆˆ
()()
n
ii
pp
i
Yb bX bX
bb
λ
=
−− −− +++
∑
λ
的值确定了对系数的惩罚强度，这个值越大，模型对数据过拟合的可能性就越小。
Lasso
也是同样的原理，只是它使用曼哈顿距离作为惩罚项，而不是欧氏距离：
2
01 1
1
ˆˆ ˆˆˆ
()
(|
||
|)
n
ii
pp
i
Yb bX bX
bb
α
=
−− −− +++
∑
如果使用欧氏距离，就称为
L2
正则化；使用曼哈顿距离，就称为
L1
正则化。在
xgboost
中，参数
lambda
（
reg_lambda
）和
alpha
（
reg_alpha
）的作用也是一样的。
6.4.4
超参数与交叉验证
xgboost
中有非常多的超参数，参见后面“
XGBoost
超参数”部分的介绍。正如在
6.4.3
节
中看到的，超参数的具体选择可以极大地改变模型拟合。有很多超参数组合可供选择，那
统计机器学习
｜
229
么应该按照什么原则来进行选择呢？这个问题的一种标准解决方案就是使用交叉验证 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.