February 2024
Intermediate to advanced
370 pages
7h
Chinese
Content preview from 数据工程之道：设计和构建健壮的数据系统
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
166
|
第
5
章
流分区的一个关键问题是确保你的分区键不会产生
热点
，即交付给一个分区的消息数量
不平均。例如，如果已知每个物联网设备位于美国的一个特定的州，我们可以使用该州
作为分区键。如果分区分布与州人口成正比，包含加利福尼亚、得克萨斯、佛罗里达和
纽约的分区可能会被数据淹没，而其他分区的利用率相对较低。确保你的分区键将在各
分区之间均匀地分配信息。
容错性和弹性
。事件流平台是典型的分布式系统，数据流存储在不同的节点上。如果一
个节点坏了，另一个节点会取代它，而流仍然可以访问。这意味着记录不会丢失。你可
以选择删除记录，但这是另一个故事。当你需要一个能够可靠地产生、存储和获取事件
数据的系统时，流媒体平台的容错性和弹性使它成为一个不错的选择。
5.4
你和谁一起工作
在访问源系统时，了解与你一起工作的人是至关重要的。根据我们的经验，与源系统的
利益相关者的良好外交和关系，往往是成功的数据工程被忽视的一个关键点。
这些利益相关者是谁？通常情况下，你会和两类利益相关者打交道：系统和数据利益相
关者（如图
5-12
所示）。
系统利益相关者
建立和维护源系统，他们可能是软件工程师、
应用程序开发者和第三方。数据利益相关者拥有并控制你对想要的数据的访问，一般由
IT
部门、数据治理小组或第三方处理。系统和数据利益相关者通常是不同的人或团队。
但有时他们也可能是相同的。
系统利益
相关者
数据利益
相关者
数据
工程师
图 5-12：数据工程师的上游利益相关者
你工作的开展常常受制于利益相关者是否有能力遵循正确的软件工程、数据库管理和开
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.