Skip to Content
面向数据科学家的实用统计学
book

面向数据科学家的实用统计学

by Peter Bruce, Andrew Bruce
October 2018
Beginner to intermediate
238 pages
6h 32m
Chinese
Posts & Telecom Press
Content preview from 面向数据科学家的实用统计学
40
|
第
2
章
假设有人说他能做到抛硬币连续十次正面向上。我们想要挑战他,这就相当于做一次实
验。如果他继续抛十次硬币,依然连续正面向上,显然这只能归因于他具有某种特异功
能,因为抛硬币连续十次正面向上的概率大约是千分之一。
现在,假设在一个体育场中有两万名观众,我们通过播音员要求全体两万人一起抛十次硬
币。如果有人做到了连续十次正面向上,就站出来。我们会看到,整个体育场中很可能
有
人
会做到连续十次正面向上。这一事件的概率非常高,甚至会高于
99%
,即
1
减去没有人
得到十次正面向上的概率。显然,我们事后从所有人中选取能做到十次正面向上的人,并
不意味着他们具有任何特异功能,这更像是运气使然。
反复地查看大规模数据集是数据科学中的一个关键价值主张,所以我们需要关注选择偏差
问题。数据科学家特别关注的一种选择偏差形式,就是被约翰
•
埃德(
John Elder
)称为
大规模搜索效应
的问题。约翰
•
埃德是美国
Elder
研究机构的创始人,该机构是一家广受
关注的数据挖掘咨询公司。如果在大规模数据集上反复运行不同的模型,并提出不同的问
题,我们肯定能发现一些有意思的现象。但是我们所发现的结果是否的确具有意义?还是
仅是一些离群值?
为了避免这一问题,我们可以使用验证集(
holdout set
)去验证结果的性能,有时可能需
要多个验证集。埃德倡议使用一种被称为
目标混洗
(
target shuffle
)的方法。该方法在本质
上就是一种置换检验,验证由数据挖掘模型所预测的关联关系的合法性。
在统计学中,除了大规模搜索效应之外,选择偏差的典型形式还包括 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

AirBnbBlueOriginElectronic ArtsHomeDepotNasdaqRakutenTata Consultancy Services

QuotationMarkO’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.
Julian F.
Head of Cybersecurity
QuotationMarkI wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.
Addison B.
Field Engineer
QuotationMarkI’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.
Amir M.
Data Platform Tech Lead
QuotationMarkI'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
Mark W.
Embedded Software Engineer

You might also like

基于Python的智能文本分析

基于Python的智能文本分析

Benjamin Bengfort, Rebecca Bilbro, Tony Ojeda
Python数据处理

Python数据处理

Jacqueline Kazil, Katharine Jarmul
精通機器學習

精通機器學習

Aurélien Géron
C++语言导学(原书第2版)

C++语言导学(原书第2版)

本贾尼 斯特劳斯特鲁普

Publisher Resources

ISBN: 9787115493668