August 2022
Intermediate to advanced
441 pages
11h 26m
Chinese
Content preview from Python文本分析
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
文本摘要
｜
289
情况下可能非常短的帖子也会包含到摘要中。如果很短的帖子中包含的单词与话题
标题相同，就会出现这种情况。这对于摘要没有任何帮助，为了纠正这个问题，我
们可以设置所有包含单词数量少于或等于
20
个的帖子不应放入摘要中：
train_df.loc[train_df['text'].str.len() <= 20, 'summaryPost'] = False
9.6.2
第二步：添加特征来辅助模型预测
由于在这个案例中，我们处理的是论坛的话题，因此我们可以生成额外的特征来帮
助模型的预测。话题的标题间接地传达了主题，有助于识别某个帖子是否应该被选
入摘要。我们不能直接将标题作为特征，因为话题中每个帖子的标题都一样，但是
我们可以计算帖子与标题之间的相似性，并将其作为特征之一：
train_df['titleSimilarity'] = train_df.apply(
lambda x: textdistance.jaro_winkler(x.text, x.Title), axis=1)
另一个有用的特征是帖子的长度。篇幅很短的帖子可能是在要求澄清问题，其中不
会包含与话题有关的最有用的信息。帖子越长，代表分享的有用信息就越多。帖子
在话题中出现的位置也可能有助于判断它是否应该在摘要中。根据论坛组织话题的
方式，帖子的位置可能有很大不同。在这个旅行论坛的例子中，帖子是按照时间顺
序排列的，其出现位置由
postNum
列指定，因此我们可以将其作为一个特征：
# Adding post length ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.