May 2024
Intermediate
252 pages
5h 31m
Chinese
Content preview from Ray 分布式机器学习：利用Ray 进行大模型的数据处理、训练、推理和部署
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
利用
Ray RLlib
进行强化学习
|
105
使用
RLlib
，用户可以完全自定义底层使用的模型和策略。如果你有深度学习的
使用经验，就会明白模型架构是多么重要。在强化学习中，模型架构通常不像
监督学习那样关键，但仍然是训练算法的重要部分。
用户还可以提供自定义的预处理器以修改观测数据的预处理方式。在简单迷宫
示例中，无须数据预处理，但在处理图像或视频数据时，预处理则是关键步骤。
在
AdvancedEnv
中，我们介绍了要避免的状态。智能体必须学会避免这些状态，
但是
RLlib
支持通过参数化动作空间自动避免惩罚。简略地说，智能体可以在
每个时间点上从动作空间中“屏蔽掉”所有要规避的动作。在某些情况下，还
可能需要具有可变的观测空间，
RLlib
也支持该功能。
我们在
4.5.3
节只是简要介绍了离线数据。
RLlib
有一个完整的
Python API
，用
于读写可以在各种情况下使用的经验数据。
简单起见，我们只使用了
DQN
算法，但是
RLlib
具有功能强大、种类众多的训
练算法。仅举一例，
MARWIL
算法是一个复杂的混合算法，可以使用它从离线
数据进行模仿学习，同时还可以混合使用生成的“在线”数据进行常规训练。
4.6
总结
本章介绍了一些有趣的
RLlib
功能，包括训练多智能体环境、使用另一智能体
生成的离线数据、设置客户
-
服务器架构将模拟与强化学习训练分离以及使用
课程学习指定越来越困难的任务。
本章还简要介绍了
RLlib
的主要概念以及如何使用其
CLI
和
Python API
。特别
地，我们展示了如何根据需求配置
RLlib
的算法和环境。本章只涵盖了 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.