Skip to Content
面向数据科学家的实用统计学
book

面向数据科学家的实用统计学

by Peter Bruce, Andrew Bruce
October 2018
Beginner to intermediate
238 pages
6h 32m
Chinese
Posts & Telecom Press
Content preview from 面向数据科学家的实用统计学
158
|
第
5
章
数据生成
类似于自助法,只是每个新的自助记录与原记录略有不同。
z
分数
对结果做归一化所生成的值。
K
在最近邻计算中使用的近邻个数。
5.5.1
 欠采样
如果有足够多的数据,就像贷款数据的例子中那样,为了使要建模的数据在
0
和
1
之间取
得平衡,一种解决方法是对多数类做
欠采样
(或下采样)。欠采样的基本思想是,认为多
数类的数据中存在很多冗余的记录。处理规模更小、更平衡的数据集,将有利于改进模型
的性能,准备数据以及探索和实验模型也会更容易。
那么多少数据是足够多的?这取决于应用,但一般来说,如果稀有类具有上万条记录,就
是足够多的。
1
越容易从
0
中区分开来,那么所需的数据就越少。
5.3
节中分析的贷款数据就是基于一个平衡的训练集,其中有一半的贷款被还清,而另一
半贷款被拖欠。预测值也给出了类似的结果,有一半预测的概率小于
0.5
,另一半预测的
概率大于
0.5
。但是在完整的数据集中,只有约
18.9%
的贷款被拖欠。
mean(loan_data$outcome == "default")
[1] 0.5
mean(full_train_set$outcome=='default')
[1] 0.1889455
如果我们使用整个数据集去训练模型,会发生什么情况?
full_model <- glm(outcome ~ payment_inc_ratio + purpose_ +
home_ + emp_len_+ dti + revol_bal + revol_util,
data=train_set, ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

AirBnbBlueOriginElectronic ArtsHomeDepotNasdaqRakutenTata Consultancy Services

QuotationMarkO’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.
Julian F.
Head of Cybersecurity
QuotationMarkI wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.
Addison B.
Field Engineer
QuotationMarkI’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.
Amir M.
Data Platform Tech Lead
QuotationMarkI'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
Mark W.
Embedded Software Engineer

You might also like

基于Python的智能文本分析

基于Python的智能文本分析

Benjamin Bengfort, Rebecca Bilbro, Tony Ojeda
Python数据处理

Python数据处理

Jacqueline Kazil, Katharine Jarmul
精通機器學習

精通機器學習

Aurélien Géron
C++语言导学(原书第2版)

C++语言导学(原书第2版)

本贾尼 斯特劳斯特鲁普

Publisher Resources

ISBN: 9787115493668