Skip to Content
Kafka权威指南
book

Kafka权威指南

by Neha Narkhede, Gwen Shapira, Todd Palino
January 2018
Beginner to intermediate
234 pages
7h 8m
Chinese
Posts & Telecom Press
Content preview from Kafka权威指南
跨集群数据镜像
127
这种方案非常复杂,我认为并不值得投入额外的时间。在索引还没有出现之前,或许可
以考虑使用这种方案。但在今天,我倾向于将集群升级到新版本,并使用基于时间戳的
解决方案,而不是进行偏移量映射,更何况偏移量映射并不能覆盖所有的失效备援场景。
3.
在失效备援之后
假设失效备援进行得很顺利,灾备集群也运行得很正常,现在需要对主集群做一些改动,
比如把它变成灾备集群。
如果能够通过简单地改变镜像进程的方向,让它将数据从新的主集群镜像到旧的主集群上
面,事情就完美了!不过,这里还存在两个问题。
怎么知道该从哪里开始镜像?我们同样需要解决与镜像程序里的消费者相关的问题。而
且不要忘了,所有的解决方案都有可能出现重复数据或者丢失数据,或者两者兼有。
之前讨论过,旧的主集群可能会有一些数据没有被镜像到灾备集群上,如果在这个时
候把新的数据镜像回来,那么历史遗留数据还会继续存在,两个集群的数据就会出现
不一致。
基于上述的考虑,最简单的解决方案是清理旧的主集群,删掉所有的数据和偏移量,然后
从新的主集群上把数据镜像回来,这样可以保证两个集群的数据是一致的。
4.
关于集群发现
在设计灾备集群时,需要考虑一个很重要的问题,就是在发生失效备援之后,应用程序
需要知道如何与灾备集群发起通信。不建议把主集群的主机地址硬编码在生产者和消费
者的配置属性文件里。大多数组织为此创建了
DNS
别名,将其指向主集群,一旦发生紧
急情况,可以将其指向灾备集群。有些组织则使用服务发现工具,比如
Zookeeper
Etcd
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

AirBnbBlueOriginElectronic ArtsHomeDepotNasdaqRakutenTata Consultancy Services

QuotationMarkO’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.
Julian F.
Head of Cybersecurity
QuotationMarkI wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.
Addison B.
Field Engineer
QuotationMarkI’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.
Amir M.
Data Platform Tech Lead
QuotationMarkI'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
Mark W.
Embedded Software Engineer

You might also like

精通機器學習

精通機器學習

Aurélien Géron
精通特征工程

精通特征工程

Alice Zheng, Amanda Casari
高效能PYTHON程式設計

高效能PYTHON程式設計

Micha Gorelick, Ian Ozsvald

Publisher Resources

ISBN: 9787115473271