August 2022
Intermediate to advanced
441 pages
11h 26m
Chinese
Content preview from Python文本分析
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
准备统计和机器学习的文本数据
｜
107
我们的最终目标是，利用处理后的文本创建一个数据库，用于分析和机器学习。因此，
每个项目的处理过程不尽相同。你应该根据具体问题来判断你的流水线中应该包含
下述哪些案例。
4.3
数据集介绍：
Reddit Self-Posts
在面对用户生成的内容时，文本数据的处理工作尤为困难。与专业报道、新闻和博
客中精心编写的文本相比，社交媒体中用户贡献的文章一般都比较短，而且包含很
多缩写、主题标签、表情符号和错别字。因此，我们将使用
Reddit Self-Posts
数据
集（
https://oreil.ly/0pnqO
），该数据集托管在
Kaggle
上。完整的数据集包含大约
100
万个带有标题和内容的用户帖子，来自
1,013
个不同的子论坛，每个子论坛有
1,000
条记录。我们使用的子集只包含其中一个类别“
autos
”（汽车），大约为
2
万个帖子。本章我们处理过的数据集将作为第
10
章单词嵌入分析的基础。
4.3.1
将数据加载到
Pandas
原始数据集由两个单独的
CSV
文件组成，一个包含帖子本身，而另一个则包含子论
坛的一些元数据，比如类别信息等。这两个文件都可以通过
pd.read_csv()
加载到
Pandas
的数据框中，然后融合成一个数据框。
import pandas as pd
posts_file = "rspct.tsv.gz"
posts_df = pd.read_csv(posts_file, sep='\t')
subred_file = "subreddit_info.csv.gz" ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.