August 2022
Intermediate to advanced
441 pages
11h 26m
Chinese
Content preview from Python文本分析
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
336
｜
第
11
章
向量化表示。我们指定参数
min_df
为
10
，且不使用二元语法。另外，我们前面的
步骤已经删除了停用词，因此向量化期间不需要考虑这个问题。我们使用同一个向
量化器来转换测试数据集，以便在评估阶段使用：
from sklearn.feature_extraction.text import TfidfVectorizer
tfidf = TfidfVectorizer(min_df = 10, ngram_range=(1,1))
X_train_tf = tfidf.fit_transform(X_train)
X_test_tf = tfidf.transform(X_test)
11.6.4
第四步：训练机器学习模型
我们曾在第
6
章中介绍过，在处理文本数据方面，支持向量机（
SVM
）是首选的机
器学习算法。众所周知，
SVM
非常善于处理拥有大量数值型特征的数据集，尤其是
我们使用的
LinearSVC
的速度非常快。此外，我们也可以选择基于树的方法，比如
随机森林或
XGBoost
，但是根据我们的经验，
LinearSVC
的准确率并不差，而且训
练非常快，因此可以更快地进行实验：
from sklearn.svm import LinearSVC
model1 = LinearSVC(random_state=42, tol=1e-5)
model1.fit(X_train_tf, Y_train)
输出结果：
LinearSVC(C=1.0, class_weight=None, dual=True, ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.