July 2014
Beginner to intermediate
456 pages
11h 26m
Japanese
Content preview from 戦略的データサイエンス入門 ―ビジネスに活かすコンセプトとテクニック
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
284
10 章 テキスト表現とテキストマイニング
は、
100
個のドキュメントからなるコーパスを扱った場合の結果を表示しています。見ての
通り、ある用語がレアであるほど（グラフ上、左に寄るほど）、
IDF
の値も高くなっていま
す。一方で、
t
がドキュメント内でより一般的になっていくと、
IDF
の値は急速に小さくなっ
ていき、漸近線の
1.0
に近づいています。ストップワードの多くは、とても一般的なので、
IDF
の値も
1.0
に近くなるでしょう。
10.3.4
手法の組み合わせ：
TFIDF
テキストの表現方法として、用語出現頻度（
Term Frequency
：
TF
）と逆文書頻度
（
Inverse Document Frequency
：
IDF
）を組み合わせた、
TFIDF
と呼ばれる表現が広く知ら
れています。あるドキュメント
d
における用語
t
の
TFIDF
の値は次の式で求めることがで
きます。
TFIDF(t, d)
＝
TF(t, d)
×
IDF(t)
IDF
がコーパス全体に対する値となるのに対して、
TFIDF
は単一のドキュメントに対する
値となることに注意してください。
bag-of-words
を用いる場合、通常はステミングとストッ
プワードの除去を行った後に用語の数をカウントします。ドキュメント内の用語数は
TF
値
を生成し、コーパス内のドキュメント数は
IDF
値を生成します。
こうして、ドキュメントはそれぞれ特徴ベクトルに変換され、コーパスは特徴ベクトルの
集合となります。そしてここでできた集合は、分類、クラスタリング、検索といった、デー ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.