August 2022
Intermediate to advanced
441 pages
11h 26m
Chinese
Content preview from Python文本分析
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
准备统计和机器学习的文本数据
｜
123
4.6.2
处理文本
流水线的执行只需要调用
nlp
对象。该调用将返回一个类型为
spacy.tokens.doc.
Doc
的对象，通过该对象可以访问词语、范围（词语的范围）以及语言方面的注释。
nlp = spacy.load("en_core_web_sm")
text = "My best friend Ryan Peters likes fancy adventure games."
doc = nlp(text)
spaCy
是面向对象的，同时也是非破坏性的。原始文本始终会保留下来。输出
doc
对象时会打印出
doc.text
的内容，该属性包含原始文本。但是，
doc
也是词语的容器，
你可以将其作为词语的迭代器：
for token in doc:
print(token, end="|")
输出结果：
My|best|friend|Ryan|Peters|likes|fancy|adventure|games|.|
实际上，每个词语都是
spaCy
的类
Token
的一个对象。词语以及文档都拥有许多有
趣的语言处理属性。表
4-2
列举了各个流水线组件会创建的属性。
注
7
表 4-2：spaCy 内置的流水线创建的属性
组件
创建
分词器
Token.is_punct
，
Token.is_alpha
，
Token.like_email
，
Token.like_url
词性标注器
Token.pos_
依赖解析器
Token.dep_
，
Token.head
，
Doc.sents
，
Doc.noun_chunks ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.