January 2018
Beginner to intermediate
234 pages
7h 8m
Chinese
Content preview from Kafka权威指南
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
构建数据管道
｜
117
7.4.2
基于图形界面的
ETL
工具
从保守的
Informatica
到一些开源的替代方案，比如
Talend
和
Pentaho
，或者更新的
Apache
NiFi
和
StreamSets
——这些
ETL
解决方案都支持将
Kafka
作为数据源和数据池。如果你已
经使用了这些系统，比如
Pentaho
，那么就可能不会为了
Kafka
而在系统里增加另一种集
成工具。如果你已经习惯了基于图形界面的
ETL
数据管道解决方案，那就继续使用它们。
不过，这些系统有一些不足的地方，那就是它们的工作流比较复杂，如果你只是希望从
Kafka
里获取数据或者将数据写入
Kafka
，那么它们就显得有点笨重。我们在本章的开头部
分已经说过，在进行数据集成时，应该将注意力集中在消息的传输上。因此，对于我们来
说，大部分
ETL
工具都太过复杂了。
我们极力建议将
Kafka
当成是一个支持数据集成（使用
Connect
）、应用集成（使用生产者
和消费者）和流式处理的平台。
Kafka
完全可以成为
ETL
工具的替代品。
7.4.3
流式处理框架
几乎所有的流式处理框架都具备从
Kafka
读取数据并将数据写入外部系统的能力。如果你
的目标系统支持流式处理，并且你已经打算使用流式框架处理来自
Kafka
的数据，那么使
用相同的框架进行数据集成看起来是很合理的。这样可以省掉一个处理步骤（不需要保存
来自
Kafka
的数据，而是直接从
Kafka
读取数据然后写到其他系统），不过在发生数据丢失
或者出现脏数据时，诊断问题会变得很困难，因为这些框架并不知道数据是什么时候丢失 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.