May 2024
Intermediate
252 pages
5h 31m
Chinese
Content preview from Ray 分布式机器学习：利用Ray 进行大模型的数据处理、训练、推理和部署
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
20
|
第
1
章
型。在原型中，通常使用简单的
HTTP
服务器，此外也有许多专门用于机器
学习模型部署的软件包。
这个列表并不完整，构建机器学习应用还涉及许多其他内容
注
13
。但是，可以肯
定的是，这
4
个步骤对于使用机器学习的数据科学项目成功与否至关重要。
1
Ray
为这
4
个与机器学习息息相关的步骤提供了专门的库。具体而言，你可以
使用
Ray Dataset
处理数据，使用
Ray Train
进行分布式模型训练，使用
Ray
RLlib
运行强化学习计算任务，使用
Ray Tune
高效调优超参数，并使用
Ray
Serve
部署模型。而且，
Ray
在设计这
4
个组件时也是基于分布式理念的。
另外，所有这些步骤都从属于训练模型的过程，极少单独使用。你希望
Ray
的
库不仅支持无缝协同工作，还能使用高度一致的
API
，因此极具优势。
Ray AIR
就是为此设计的，它能提供统一的运行时和
API
，还能随时进行扩展。图
1.2
展
示了
AIR
的所有组件。
数据处理工具
Ray Dataset
模型训练工具
Ray Train
Ray RLlib
超参数调优工具
Ray Tune
模型部署工具
Ray Serve
Ray AI Runtime
（
AIR
）
图 1.2：Ray AIR 的组件
虽然本章篇幅有限，无法详细介绍
Ray AIR
的
API
（详见第
10
章），但会介绍
Ray AIR
中的所有组件。
1.3.2
处理数据
我们首先介绍
Ray Datasets
。该库包含一个名为
Dataset
的数据结构、多种用于
从各种格式和系统加载数据的连接器、用于转换数据集的
API
，以及使用它们 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.