January 2018
Beginner to intermediate
234 pages
7h 8m
Chinese
Content preview from Kafka权威指南
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
116
｜
第
7
章
转换器将各种格式（
Avro
、
JSON
或
String
）的数据转换成
Data API
格式的对象，然后将
它们传给目标连接器，目标连接器再将它们插入到目标系统。
Connect API
因此可以支持多种类型的数据，数据类型与连接器的实现是相互独立的——
只要有可用的转换器，连接器和数据类型可以自由组合。
4.
偏移量管理
worker
进程的
REST API
提供了部署和配置管理服务，除此之外，
worker
进程还提供了
偏移量管理服务。连接器只要知道哪些数据是已经被处理过的，就可以通过
Kafka
提供的
API
来维护偏移量。
源连接器返回给
worker
进程的记录里包含了一个逻辑分区和一个逻辑偏移量。它们并非
Kafka
的分区和偏移量，而是源系统的分区和偏移量。例如，对于文件源来说，分区可以
是一个文件，偏移量可以是文件里的一个行号或者字符号；而对于
JDBC
源来说，分区可
以是一个数据表，偏移量可以是一条记录的主键。在设计一个源连接器时，要着重考虑如
何对源系统的数据进行分区以及如何跟踪偏移量，这将影响连接器的并行能力，也决定了
连接器是否能够实现至少一次传递或者仅一次传递。
源连接器返回的记录里包含了源系统的分区和偏移量，
worker
进程将这些记录发送给
Kafka
。如果
Kafka
确认记录保存成功，
worker
进程就把偏移量保存下来。偏移量的存储
机制是可插拔的，一般会使用
Kafka
主题来保存。如果连接器发生崩溃并重启，它可以从
最近的偏移量继续处理数据。
目标连接器的处理过程恰好相反，不过也很相似。它们从 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.