January 2018
Beginner to intermediate
234 pages
7h 8m
Chinese
Content preview from Kafka权威指南
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
深入
Kafka
｜
83
因为在一个大文件里查找和删除消息是很费时的，也很容易出错，所以我们把分区分成若
干个
片段
。默认情况下，每个片段包含
1GB
或一周的数据，以较小的那个为准。在
broker
往分区写入数据时，如果达到片段上限，就关闭当前文件，并打开一个新文件。
当前正在写入数据的片段叫作
活跃片段
。活动片段永远不会被删除，所以如果你要保留数
据
1
天，但片段里包含了
5
天的数据，那么这些数据会被保留
5
天，因为在片段被关闭之
前这些数据无法被删除。如果你要保留数据一周，而且每天使用一个新片段，那么你就会
看到，每天在使用一个新片段的同时会删除一个最老的片段——所以大部分时间该分区会
有
7
个片段存在。
我们在第
2
章讲过，
broker
会为分区里的每个片段打开一个文件句柄，哪怕片段是不活跃
的。这样会导致打开过多的文件句柄，所以操作系统必须根据实际情况做一些调优。
5.5.3
文件格式
我们把
Kafka
的消息和偏移量保存在文件里。保存在磁盘上的数据格式与从生产者发送过
来或者发送给消费者的消息格式是一样的。因为使用了相同的消息格式进行磁盘存储和网
络传输，
Kafka
可以使用零复制技术给消费者发送消息，同时避免了对生产者已经压缩过
的消息进行解压和再压缩。
除了键、值和偏移量外，消息里还包含了消息大小、校验和、消息格式版本号、压缩算法
（
Snappy
、
GZip
或
LZ4
）和时间戳（在
0.10.0
版本里引入的）。时间戳可以是生产者发送消
息的时间，也可以是消息到达
broker
的时间，这个是可配置的。
如果生产者发送的是压缩过的消息，那么同一个批次的消息会被压缩在一起，被当作 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.