October 2014
Intermediate to advanced
288 pages
6h 27m
Japanese
Content preview from 実践 機械学習システム
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
48
3
章 関連のある文書を見つける
3.2
前処理：共通する単語の出現回数を類似度として計測する
これまで見てきたとおり、bag-of-wordは高速であり、かつロバストです。しかし、問題がないわ
けではありません。それでは、その問題について見ていくことにしましょう。
3.2.1
テキストデータを
bag-of-word
に変換する
ここでは、単語の出現回数を数え、それをベクトルで表記するためのコードを書く必要はありませ
ん。Scikitの
CountVectorizer
がその作業を効率的に行ってくれます。また、それには便利なイン
ターフェイスが備わっています。Scikit の関数とクラスは
sklearn
パッケージを経由して、次のよう
にインポートすることができます。
>>>
from sklearn.feature_extraction.text import CountVectorizer
>>>
vectorizer = CountVectorizer(min_df=1)
上で使用した
min_df
というパラメータは、頻繁には使われていない単語を、
CountVectorizer
が無視するときに使用します。もし、設定する値が整数であれば、その数より出現回数の小さい単語
は無視されます。設定する値が分数であれば、データセットの数にその分数を掛けた値を基準とし
て、その基準より出現回数が小さい場合、それは無視されます。
max_df
というパラメータも同様な働
きをします。生成したインスタンスを
print
すること
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.