January 2018
Beginner to intermediate
234 pages
7h 8m
Chinese
Content preview from Kafka权威指南
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
流式处理
｜
191
11.3.1
单个事件处理
处理单个事件是流式处理最基本的模式。这个模式也叫
map
或
filter
模式，因为它经常被
用于过滤无用的事件或者用于转换事件（
map
这个术语是从
Map-Reduce
模式中来的，
map
阶段转换事件，
reduce
阶段聚合转换过的事件）。
在这种模式下，应用程序读取流中的事件，修改它们，然后把事件生成到另一个流上。比
如，一个应用程序从一个流中读取日志消息，并把
ERROR
级别的消息写到高优先级的流
中，同时把其他消息写到低优先级的流中。再如，一个应用程序从流中读取事件，并把事
件从
JSON
格式改为
Avro
格式。这类应用程序不需要在程序内部维护状态，因为每一个
事件都是独立处理的。这也意味着，从错误中恢复或进行负载均衡会非常容易，因为不需
要进行恢复状态的操作，只需要将事件交给应用程序的另一个实例去处理。
这种模式可以使用一个生产者和一个消费者来实现，如图
11-3
所示。
主题
日志事件
分支
低优
先级
主题
错误事件
其他事件
高优
先级
主题
转换成
Avro
Avro
日志
图 11-3：单事件处理拓扑
11.3.2
使用本地状态
大部分流式处理应用程序关心的是如何聚合信息，特别是基于时间窗口进行聚合。例如，
找出每天最低和最高的股票交易价格并计算移动平均数。
要实现这些聚合操作，需要维护流的
状态
。在本例中，为了计算每天的最小价格和平均价
格，需要将最小值和最大值保存下来，并将它们与每一个新值进行对比。
这些操作可以通过
本地
状态（而不是共享状态）来实现，因为本例中的每一个操作都是基
于组的聚合操作，如图 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.