November 2021
Intermediate to advanced
302 pages
8h 57m
Chinese
Content preview from 机器学习流水线实战
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
流水线第二部分：
Kubeow Pipelines
｜
221
一种选择是使用
Google Cloud Dataflow
执行
Apache Beam
。在这种情况下，
TFX
将使用
Apache Beam
处理作业
，后者将向
Dataflow
提交任务。根据每个作业的要求，
Dataflow
将
启动计算实例并在各个实例之间分配作业任务。这是扩展数据预处理作业（比如统计信息
生成或数据预处理）的一种很巧妙的方法。
5
为了使用
Google Cloud Dataflow
的扩展功能，需要提供更多的
Apache Beam
配置，并将其
传给流水线做实例化：
tmp_file_location = os.path.join(output_bucket, "tmp")
beam_pipeline_args = [
"--runner=DataflowRunner",
"--experiments=shuffle_mode=auto",
"--project={}".format(project_id),
"--temp_location={}".format(tmp_file_location),
"--region={}".format(gcp_region),
"--disk_size_gb=50"
]
除了将
DataflowRunner
配置为
runner
类型，我们还将
shuffle_mode
设置为了
auto
。这是
Dataflow
一个有趣的功能
。当运行诸如
GroupByKey
之类的转换时，该操作将在
Dataflow ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.