Skip to Content
Kafka权威指南(第2版)
book

Kafka权威指南(第2版)

by Gwen Shapira, Todd Palino, Rajini Sivaram, Krit Petty
November 2022
Beginner to intermediate
346 pages
11h
Chinese
Posts & Telecom Press
Content preview from Kafka权威指南(第2版)
精确一次性语义
139
8.2.1
 事务的应用场景
一些流式处理应用程序对准确性要求较高,特别是如果处理过程包含了聚合或连接操作,
那么事务对它们来说就会非常有用。如果流式处理应用程序只进行简单的转换和过滤,那
么就不需要更新内部状态,即使出现了重复消息,也可以很容易地将它们过滤掉。但是,
如果流式处理应用程序对几条消息进行了聚合,一些输入消息被统计了不止一次,那么就
很难知道结果是不是错误的。如果不重新处理输入消息,则不可能修正结果。
金融行业的应用程序就是典型的复杂流式处理的例子,在这些应用程序中,精确一次性被
用于保证精确的聚合结果。不过,因为可以非常容易地在
Streams
应用程序中启用精确一
次性保证,所以已经有非常多的应用场景(如聊天机器人)启用了这个特性。
8.2.2
 事务可以解决哪些问题
假设有一个简单的流式处理应用程序:它从源主题读取消息,然后可能会对消息做一些处
理,再将结果写入另一个主题。我们想要确保处理的每一条消息的结果只被写入一次。那
么,哪些地方有可能出错呢?
事实证明,很多地方有可能出错。下面来看看其中的两种情况。
1.
应用程序崩溃导致的重复处理
在从源集群读取并处理了消息之后,应用程序必须做两件事:一是将结果写入输出主题,
二是提交已处理的消息的偏移量。假设这两个动作就按照这个顺序发生。如果应用程序在
发送结果之后发生崩溃,但偏移量还没有提交,该怎么办?
4
章讨论过当消费者崩溃时会发生什么
。几秒之后,因为没有心跳,所以将触发再均
衡,消费者读取的分区将被重新分配给其他消费者。新消费者将从最后提交的偏移量的位 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

AirBnbBlueOriginElectronic ArtsHomeDepotNasdaqRakutenTata Consultancy Services

QuotationMarkO’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.
Julian F.
Head of Cybersecurity
QuotationMarkI wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.
Addison B.
Field Engineer
QuotationMarkI’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.
Amir M.
Data Platform Tech Lead
QuotationMarkI'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
Mark W.
Embedded Software Engineer

You might also like

时间序列分析实战:基于机器学习和统计学

时间序列分析实战:基于机器学习和统计学

Aileen Nielsen
Spark机器学习实战

Spark机器学习实战

Posts & Telecom Press, Siamak Amirghodsi, Meenakshi Rajendran, Broderick Hall, Shuen Mei
写给系统管理员的Python脚本编程指南

写给系统管理员的Python脚本编程指南

Posts & Telecom Press, Ganesh Sanjiv Naik
Kubernetes编程

Kubernetes编程

Michael Hausenblas, Stefan Schimanski

Publisher Resources

ISBN: 9787115601421