May 2018
Intermediate to advanced
228 pages
4h 11m
Japanese
Content preview from Rによるテキストマイニング ―tidytextを活用したデータ分析と可視化の基礎
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
38
■
3
章 単語の出現頻度と特定の文書での出現頻度の分析：
tf-idf
図3-1 ジェーン・オースティンの小説における単語出現頻度の分布
どの小説でも、右の方に非常に長い裾、すなわちロングテールが形成されていま
す（非常に出現頻度の高いありふれた単語）。実は、裾の先端はこのグラフには描か
れていません。どの小説でも、グラフが示す分布はほぼ同じです。出現頻度が非常
に低い多数の単語があり、出現頻度が高い少数の単語があります。
3.2
ジップの法則
言語の世界では、図 3-1 のような分布はごく一般的です。実際、自然言語のどの
コーパスでも（書籍、
Web
サイトのテキスト、話し言葉など）、このような長い裾を
持つ分布は非常に一般的なので、単語の出現頻度（
frequency
）と出現頻度順位（
rank
）
の関係は、以前から研究対象になっています。この関係の古典的なものは、
20
世紀
3.2
ジップの法則
■
39
アメリカの言語学者、ジョージ・ジップ（
George Zipf
）にちなんでジップの法則と呼
ばれています。
ジップの法則では、単語の出現頻度は出現頻度順位に反比例すると主張
されています。
単語出現頻度のグラフを描けるデータフレームはすでにあるので、
dplyr
関数を
使った数行でジェーン・オースティンの小説でジップの法則を検証してみましょう。 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.