August 2022
Intermediate to advanced
441 pages
11h 26m
Chinese
Content preview from Python文本分析
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
134
｜
第
4
章
4.7
大型数据集的特征提取
通过以上学习，我们掌握了
spaCy
提供的各种工具，下面我们终于可以构建自己的
语言特征提取器了。图
4-3
说明了各项工作。我们希望最终创建一个数据集，作为
统计分析和各种机器学习算法的输入。在完成数据提取后，我们将预处理的数据保
存到数据库，以方便随时使用。
清理后的
文本
spaCy
流水线
特征提取
所有词元 名词
形容词
名词短语
命名实体
处理后的
文本
图 4-3：使用 spaCy 提取文本的特征
4.7.1
案例：创建一个一体化函数
本案例中的函数涵盖了上一节所有的提取函数。这段代码将所有要提取的内容都放
到了同一个函数中，因此，即便你需要添加或修改某些内容，也不需要对后续步骤
进行任何调整：
def extract_nlp(doc):
return {
'lemmas' : extract_lemmas(doc,
exclude_pos = ['PART', 'PUNCT',
'DET', 'PRON', 'SYM', 'SPACE'],
filter_stops = False),
'adjs_verbs' : extract_lemmas(doc, include_pos = ['ADJ', 'VERB']),
'nouns' : extract_lemmas(doc, include_pos ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.