January 2017
Beginner to intermediate
310 pages
7h 4m
Chinese
Content preview from 面向机器学习的自然语言标注
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
122
｜
第
6
章
我们已经接近完成。
P
在最初的
Fleiss
的公式中表示
P
i
的平均值。因此，我们可以将
P
i
列的数值相加，并除以标注人员的数量来计算
P
值。
P
= (0.3343 + 0.3384 + 0.3384 + 0.3328 + 0.3646) / 5
=1.7085/5
=
0.3417
然后，将
P
c
行的数值的平方相加来计算
P
(
e
)
：
P(e)
= 0.3072
2
+ 0.3568
2
+ 0.336
2
= 0.335
现在，可将这些数值代入
Fleiss
的
Kappa
公式，并计算
IAA
得分：
κ
= (0.3417
－
0.335) / (1
－
0.335)
=0.0067/0.665
=
0.004
6.8.3
解释
Kappa
系数
在前面两节中，针对不同的标注任务，我们计算了两个不同的
κ
值：
0.29
和
0.004
。
但对这两个值应该怎样解释呢？在很多情况下，对
Kappa
系数的解释取决于标注任务的
复杂性和客观性。因此，并不存在一个判定这些得分优劣的硬性规则。比如，在一个词
性标注任务（可能需要进行大量的标注工作）中可能得到一个接近
1.0
的
κ
值，具体数
值取决于相关术语的定义及其理论基础。另一方面，像语义角色标注或时间标注（其中，
一个文本可能的确允许多个有效的解释）等需要对文本进行更深入理解的任务并不总是
能够保持如此高的水平。
Landis and Koch
（
1977
）为解释
κ
值及其他一致性度量提出了如下规则：
κ
一致性水平
<0
差
0.01-0.20
较差
0.21-0.40
一般
0.41-0.60
中等
0.61-0.80 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.