Skip to Content
Google SRE工作手册
book

Google SRE工作手册

by Betsy Beyer, Niall Richard Murphy, David K. Rensin, Kent Kawahara, Stephen Thorne
September 2020
Intermediate to advanced
526 pages
8h 23m
Chinese
China Electric Power Press Ltd.
Content preview from Google SRE工作手册
数据处理流水线
293
复杂业务关键型数据处理流水线。无论你是直接负责流水线,还是依赖于流水线产
生数据的其他服务,我们都希望本章中的信息可以帮助你实现更可靠的流水线(和
服务)。
有关
Google
数据处理流水线理念的全面讨论,请参阅我们的第一本
SRE
书的第
25
章。
流水线应用
流水线有各种各样的应用,每个应用都有自己的优势和适合的应用场景。流水线可
以包含多个阶段,每个阶段都是一个依赖于其他阶段的独立过程。一个流水线可能
包含的多个阶段可以抽象为高层次的范式。
Cloud Dataflow
是一个示例:用户使用相
对高层次的
API
编写业务逻辑,而流水线技术本身将此数据转换为一系列步骤或阶
段,其中的一个输出是另一个的输入。为了让你对流水线应用有更广泛的认识,接
下来我们将介绍各种流水线应用及其推荐的用法。我们例子里的两个公司有不同的
流水线和实现要求,我们将展示他们用不同的方式来满足各自的数据需求。这些示
例阐述了:如何根据你的特定用例来定义项目目标,以及如何使用这些目标来明智
地选择出最适合你的数据流水线类型。
从事件处理
/
数据转置到有序或结构化数据
萃取、转置、加载(
Extract Transform Load
ETL
)模型是数据处理中的常见方法:
从数据源中萃取、并转置(可能进行)逆规范化,然后“重新加载”为特定的格式。
在更先进的应用中,这看起来可能像是一个认知过程:数据从某种传感器(实时或
回放)中获取,经过挑选、编码阶段后,对特定的数据结构进行“训练”(如机器
学习网络)。
ETL
流水线的方式工作也很类似。 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

AirBnbBlueOriginElectronic ArtsHomeDepotNasdaqRakutenTata Consultancy Services

QuotationMarkO’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.
Julian F.
Head of Cybersecurity
QuotationMarkI wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.
Addison B.
Field Engineer
QuotationMarkI’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.
Amir M.
Data Platform Tech Lead
QuotationMarkI'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
Mark W.
Embedded Software Engineer

You might also like

Python数据分析(第2版)

Python数据分析(第2版)

Posts & Telecom Press, Armando Fandango
Google系统架构解密: 构建安全可靠的系统

Google系统架构解密: 构建安全可靠的系统

Heather Adkins, Betsy Beyer, Paul Blankinship, Piotr Lewandowski, Ana Oprea, Adam Stubblefield
编写整洁的Python代码(第2版)

编写整洁的Python代码(第2版)

Posts & Telecom Press, Mariano Anaya

Publisher Resources

ISBN: 9787519845858