May 2024
Intermediate
252 pages
5h 31m
Chinese
Content preview from Ray 分布式机器学习：利用Ray 进行大模型的数据处理、训练、推理和部署
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
166
|
第
8
章
是一个
Ray
执行器。进入的流量通过
HTTP
代理，该代理负责负载均衡，将请
求分配给副本
注
1
。
1
用户请求
HTTP 代理 HTTP 代理
Ray Cluster
节点 1节点 1
部署模块 A
副本 1
部署模块 B
副本 1
部署模块 A
副本 2
部署模块 C
副本 2
图 8.1：Ray Serve 应用程序的架构
在底层，组成
Ray Serve
应用程序的部署模块由一个集中的控制器执行器进行管
理。这是由
Ray
管理的独立执行器，可在发生故障时重新启动。控制器负责创
建和更新副本执行器，在系统中广播更新，并执行健康检查和故障恢复。如果
某个副本或整个
Ray
节点崩溃了，控制器将检测故障并确保执行器恢复，继续
提供服务。
8.2.2
定义基础
HTTP
端点
本小节通过定义一个简单的
HTTP
端点，并包装单个
ML
模型来介绍
Ray
Serve
。我们要部署的模型是一个情感分类器，即给定文本输入，判断文本情
感是积极的还是消极的。我们将使用
Hugging Face Transformers
库（
https://
huggingface.co
）提供的预训练情感分类器，该库提供了一个简单的
Python API
用于预训练模型。这个库对模型细节做了抽象，使我们能够专注于服务逻辑。
使用
Ray Serve
部署模型，我们需要定义一个
Python
类，并使用
@serve.
deployment
装饰器将其转换为
Serve
部署模块。这个装饰器支持传递一些有用
的选项来配置部署模块，
8.2.3
节将介绍一些配置选项：
注 1 ：部署模块也可以直接向彼此发送流量。这使得构建涉及模型组合，或者构建将 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.