May 2024
Intermediate
252 pages
5h 31m
Chinese
Content preview from Ray 分布式机器学习：利用Ray 进行大模型的数据处理、训练、推理和部署
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
202
|
第
10
章
用于训练模型或访问自定义数据源。与构建在
Ray Core
之上的其他机器学习库
一样，
AIR
隐藏了底层抽象，提供了直观的
API
，其灵感来自
scikit-learn
等工
具的常见模式。
Ray AIR
既服务于数据科学家又服务于机器学习工程师。数据科学家可以使用
Ray AIR
构建和扩展端到端实验或单独的子任务，如预处理、训练、调优、评
分或
ML
模型部署。机器学习工程师可以构建基于
AIR
的自定义机器学习平台，
或者简单地利用其统一的
API
将其与生态中的其他库集成。而且，
Ray
提供了
探索底层
Ray Core API
的灵活性。
作为
Ray
生态的一部分，
AIR
可以利用
Ray
的所有优势，包括从笔记本上的实
验到集群上的生产工作流的无缝过渡。通常情况下，数据科学团队需要将机器
学习代码“移交”给负责生产系统的团队。这样成本高又耗时，因为移交过程
通常涉及修改甚至重写部分代码。正如后面展示的，因为
AIR
可处理可扩展性、
可靠性和稳健性等问题，所以
Ray AIR
可以协助用户进行生产过渡。
Ray AIR
具有相当数量的集成工具，但它仍然是可扩展的。正如
10.2
节将展示
的，
AIR
的统一
API
提供了流畅的工作流，可以轻松替换其中的许多组件。例
如，你可以使用相同的接口在
AIR
中定义
XGBoost
或
PyTorch
的训练器，从而
方便地尝试不同的机器学习模型。
同时，选择
AIR
可以避免与多个分布式系统协同工作，并为它们编写复杂的粘
合代码。经常使用多个组件的团队往往会遇到集成工具的快速废弃和高维护成
本的问题。这些问题可能会导致迁移疲劳 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.