What Is Ray?

Chapter 3. Machine Learning Libraries That Use Ray

In the first chapter, we learned that Ray was created to address the need for flexible, efficient, and easy-to-use distributed computing, especially for newer machine learning systems like reinforcement learning, which are mostly written in Python. We also learned about the Ray ecosystem today.

In the last chapter, we saw Ray’s concise and intuitive but low-level API in action.

Now we’ll discuss several of the libraries made possible by Ray, which were drivers for the creation of Ray, too. They include Ray RLlib for reinforcement learning, Ray Tune for hyperparameter tuning, Ray S_GD_ for distributed training of TensorFlow and PyTorch models, and Ray Serve for model serving.

What Is Reinforcement Learning?

Reinforcement learning (RL) is a large and diverse subject. We can’t do it justice here, but we can explore the highlights and see how Ray RLlib enables RL practitioners to work efficiently. RLlib is also modular and flexible to support researchers exploring new RL algorithms and techniques.

So here goes a whirlwind tour of reinforcement learning. I’ll italicize popular terms as we encounter them. Consider Figure 3-1.

Figure 3-1. Reinforcement learning

An agent takes actions in an environment, attempting to maximize a cumulative reward. At each step, the agent observes the environment’s current state and the reward received ...