August 2022
Intermediate to advanced
441 pages
11h 26m
Chinese
Content preview from Python文本分析
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
26
｜
第
1
章
tqdm
是一个很棒的
Python
库，可用于显示进度条。它支持常规循环，
例如可以使用
tqdm_range
（代替
range
），而且还支持
Pandas
：提供了
progress_map
和
progress_apply
等数据框操作。
注
3
本书附带的
GitHub
库中的
notebook
使用了这些操作，但是在本书中，我们只使用
Pandas
。
1.6
词频分析案例
我们可以通过频繁出现的单词和短语，对讨论的主题有一些基本的理解。但是，词
频分析忽略了单词的顺序和上下文。这就是著名的“词袋模型”思想（请参见第
5
章 ）：
所有单词都被扔进一个袋子里，揉成一团。文本中原有的语序都丢失了，仅考虑词
条的频率。这种模型不适用情感分析或问题解答等复杂的任务，但对于分类和主题
建模却非常有效。此外，这也是了解文本内容的一个很好的起点。
在本节中，我们来开发一些计算和可视化词频的解决方案。由于原始频率会赋予频
繁出现、但不重要的单词过高的权重，因此我们还将在最后介绍
TF-IDF
。我们将使
用
Counter
来实现词频计算，因为它既简单又快。
1.6.1
案例：使用
Counter
统计单词个数
Python
的标准库有一个内置的
Counter
类，其功能很简单，即计数。
注
4
最简单的使用
Counter
的方法就是创建一个列表项的计数，在我们的例
子中，这个字符串列表就是单词或词语。最后得到的
Counter
是一个字典对象，其
中包含的键是列表的各项，而值就是词频。
下面，我们通过一个简单的例子来说明：
from collections import ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.