May 2024
Intermediate
252 pages
5h 31m
Chinese
Content preview from Ray 分布式机器学习：利用Ray 进行大模型的数据处理、训练、推理和部署
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
230
|
第
11
章
11.1.4
创建自定义集成
在详细解释
Ray
与其他复杂软件框架的关系之前，我们先讨论如何为
Ray AIR
创建自定义集成。由于
AIR
具有可扩展性，因此你可以找到适合创建自定义集
成中所有任务的合适接口。
1
例如，假设你想从
Snowflake
读取数据，在其上训练
JAX
模型，并将调优结果
记录到
Neptune
注
8
中。在撰写本书时，还没有这样的集成可用，但这很可能会
发生改变。我们选择这些集成（
Snowflake
、
JAX
、
Neptune
）并不是出于任何偏
好，它们只是生态中的常规工具。无论如何，了解如何创建集成都是值得的。
要将数据从
Snowflake
加载到
Ray Dataset
中，你必须创建一个新的
Data
source
。通过指定如何启动（
create_reader
），如何写入源（
do_write
），以及
在成功写入和失败写入时发生什么（
on_write_complete
和
on_write_failed
），
你可以定义一个
Datasource
。给定具体的
SnowflakeDatasource
实现，就可以
将数据读取到
Ray Dataset
中：
from ray.data import read_datasource, datasource
class SnowflakeDatasource(datasource.Datasource):
pass
dataset = read_datasource(SnowflakeDatasource(), ...)
接下来，假设你有一个
JAX
模型，你希望使用 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.