January 2018
Beginner to intermediate
234 pages
7h 8m
Chinese
Content preview from Kafka权威指南
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
58
｜
第
4
章
这些事件在进行再均衡时会
被重新处理，导致重复
正在处理的事件
上一次轮询
返回的事件
上一次提交的偏移量
图 4-6：提交的偏移量小于客户端处理的最后一个消息的偏移量
如果提交的偏移量大于客户端处理的最后一个消息的偏移量，那么处于两个偏移量之间的
消息将会丢失，如图
4-7
所示。
正在处理的事件
上一次轮询
返回的事件
这些事件在进行
再均衡时会丢失
上一次提交的偏移量
图 4-7：提交的偏移量大于客户端处理的最后一个消息的偏移量
所以，处理偏移量的方式对客户端会有很大的影响。
KafkaConsumer API
提供了很多种方式来提交偏移量。
4.6.1
自动提交
最简单的提交方式是让消费者自动提交偏移量。如果
enable.auto.commit
被设为
true
，那
么每过
5s
，消费者会自动把从
poll()
方法接收到的最大偏移量提交上去。提交时间间隔
由
auto.commit.interval.ms
控制，默认值是
5s
。与消费者里的其他东西一样，自动提交
也是在轮询里进行的。消费者每次在进行轮询时会检查是否该提交偏移量了，如果是，那
么就会提交从上一次轮询返回的偏移量。
不过，在使用这种简便的方式之前，需要知道它将会带来怎样的结果。
假设我们仍然使用默认的
5s
提交时间间隔，在最近一次提交之后的
3s
发生了再均衡，再
均衡之后，消费者从最后一次提交的偏移量位置开始读取消息。这个时候偏移量已经落后
了
3s
，所以在这
3s
内到达的消息会被重复处理。可以通过修改提交时间间隔来更频繁地
提交偏移量，减小可能出现重复消息的时间窗，不过这种情况是无法完全避免的。 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.