January 2018
Beginner to intermediate
234 pages
7h 8m
Chinese
Content preview from Kafka权威指南
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
流式处理
｜
193
11.3.3
多阶段处理和重分区
本地状态对按
组
聚合操作起到很大的作用。但如果需要使用所有可用的信息来获得一个结
果呢？例如，假设要发布每天的“前
10
支”股票，这
10
支股票需要从每天的交易股票中
挑选出来。很显然，如果只是在每个应用实例上进行处理是不够的，因为
10
支股票分布
在多个实例上，如图
11-5
所示。我们需要一个两阶段解决方案。首先，计算每支股票当天
的涨跌，这个可以在每个实例上进行。然后将结果写到一个包含了单个分区的新主题上。
另一个单独的应用实例读取这个分区，找出当天的前
10
支股票。新主题只包含了每支股
票的概要信息，比其他包含交易信息的主题要小很多，所以流量很小，使用单个应用实例
就足以应付。不过，有时候需要更多的步骤才能生成结果。
交易主题
分区
分区
处理器
处理器
处理器
本地
状态
每日获利
或损失
每日获利
或损失
获利或损失主题
一个
分区
一个
分区
前10个主题
本地
状态
本地
状态
图 11-5：包含本地状态和重分区步骤的拓扑
这种多阶段处理对于写过
Map-Reduce
代码的人来说应该很熟悉，因为他们经常要使用多
个
reduce
步骤。如果写过
Map-Reduce
代码，就应该知道，处理每个
reduce
步骤的应用需
要被隔离开来。与
Map-Reduce
不同的是，大多数流式处理框架可以将多个步骤放在同一
个应用里，框架会负责调配每一步需要运行哪一个应用实例（或
worker
）。
11.3.4
使用外部查找
——
流和表的连接
有时候，流式处理需要将外部数据和流集成在一起，比如使用保存在外部数据库里的规则
来验证事务，或者将用户信息填充到点击事件当中。 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.