January 2018
Beginner to intermediate
234 pages
7h 8m
Chinese
Content preview from Kafka权威指南
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
第 7 章
构建数据管道
在使用
Kafka
构建数据管道时，通常有两种使用场景：第一种，把
Kafka
作为数据管道的
两个端点之一，例如，把
Kafka
里的数据移动到
S3
上，或者把
MongoDB
里的数据移动
到
Kafka
里；第二种，把
Kafka
作为数据管道两个端点的中间媒介，例如，为了把
Twitter
的数据移动到
ElasticSearch
上，需要先把它们移动到
Kafka
里，再将它们从
Kafka
移动到
ElasticSearch
上。
LinkedIn
和其他一些大公司都将
Kafka
用在上述两种场景中，后来，我们在
0.9
版本的
Kafka
里增加了
Kafka Connect
（以下简称
Connect
）。我们注意到，企业在将
Kafka
与数据
管道进行集成时总会碰到一些特定的问题，所以决定往
Kafka
里增加一些
API
来帮助他们
解决这些问题，而不是等着他们提出这些问题。
Kafka
为数据管道带来的主要价值在于，它可以作为数据管道各个数据段之间的大型缓冲
区，有效地解耦管道数据的生产者和消费者。
Kafka
的解耦能力以及在安全和效率方面的
可靠性，使它成为构建数据管道的最佳选择。
数据集成的场景
有些组织把
Kafka
看成是数据管道的一个
端点
，他们会想“我怎么才能把数
据从
Kafka
移到
ElasticSearch
里”。这么想是理所当然的——特别是当你需
要的数据在到达
ElasticSearch
之前还停留在
Kafka
里的时候，其实我们也是
这么想的。不过我们要讨论的是如何在更大的场景里使用
Kafka
，这些场景
至少包含两个端点（可能会更多） ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.