September 2013
Intermediate to advanced
264 pages
5h
Chinese
Content preview from Bad Data 技術手冊
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
建立語料庫
|
91
Word Pos Neg Chi-Sq
romantic 137 117 0.1913
movies 635 571 0.0036
hair 57 52 0.0033
produced 67 61 0.0026
bob 96 86 0.0024
where 785 707 0.0013
face 188 169 0.0013
take 476 429 0.0003
你能夠看到，「good」是其中一個最常被濫用的詞，並且提供非常少的資訊。而若你想
要一個清新且強烈的觀眾反應，那麼你也不應該將電影角色的名稱命名為「Bob」。當在
訓練一個文字分類器時，這些低資訊的單字是有傷害的雜訊，這些單字應該要被丟棄或
是將其權重調低，這取決於你使用的分類演算法。簡單貝氏分類器
10
（Naive Bayes）在
資料擁有雜訊時表現特別不好，但邏輯迴歸分類器（Logistic Regression）（也被稱作最
大熵學習法，Maximum Entropy）
11
可以依照重要性決定權重，而會將雜訊的特徵值權
重調降。
建立語料庫
接下來要更詳細地介紹語料庫的建立：我們將每個評論視為是單一的
實體
。最簡單做
到的方法，就是將所有單一評論中的換行符號置換為單一空格，藉此讓每則評論都看起
來像是一篇文章。然後將每則評論∕文章用空白的行作區隔，這樣就能夠很容易的辨識
出一則評論的結束以及下一則的開始。依據每個分類的評論數量，你也許希望每個分類
都擁有多個檔案，而每個檔案包含一些合理數量且用空白行分隔的評論。因為有多個檔
案，你應該像是 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.