August 2022
Intermediate to advanced
441 pages
11h 26m
Chinese
Content preview from Python文本分析
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
128
｜
第
4
章
4.6.5
案例：根据词性提取词元
词形还原（
Lemmatization
）是将单词还原为变形前的词根的映射。在统计、
机器学习以及信息检索中，将单词“
housing
”、“
housed
”和“
house
”
当成同一个词处理有许多优势。这种方式不仅可以提高模型的质量，而且可以减少
训练时间和模型大小，因为只保留未变形的单词，可以大大减小词汇量。
另外，将
使用的单词类型限制为特定类别（比如名词、动词和形容词）通常会很有帮助。这
些单词类型称为词性标签（
part-of-speech tags
）。
下面，我们来仔细研究一下词形还原。词语或范围的词元可通过
lemma_
属性访问，
如下例所示：
text = "My best friend Ryan Peters likes fancy adventure games."
doc = nlp(text)
print(*[t.lemma_ for t in doc], sep='|')
输出结果：
-PRON-|good|friend|Ryan|Peters|like|fancy|adventure|game|.
正确指定词元需要查询词典，还需要了解有关单词词性的知识。例如，名词“
meeting
”
的词元就是“
meeting
”，而动词的词元则是“
meet
”。
spaCy
可以区分英语中的
这种情况。然而，在大多数其他语言中，词形还原完全依赖于词典，而不依赖于词
性。请注意，人称代词（比如“
I
”、“
you
”和“
her
”）在
spaCy
中的词元永远是
“
-PRON-
”。 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.