book

Learning Ray

by Max Pumperla, Edward Oakes, Richard Liaw

February 2023

Beginner

271 pages

7h 15m

English

O'Reilly Media, Inc.

Book available

Read now

Unlock full access

Who Should Read This BookGoals of This BookNavigating This BookHow to Use the Code ExamplesConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
What Is Ray?What Led to Ray?Ray’s Design PrinciplesThree Layers: Core, Libraries, and EcosystemA Distributed Computing FrameworkA Suite of Data Science LibrariesRay AIR and the Data Science WorkflowData Processing with Ray DatasetsModel TrainingHyperparameter TuningModel ServingA Growing EcosystemSummary
An Introduction to Ray CoreA First Example Using the Ray APIAn Overview of the Ray Core APIUnderstanding Ray System ComponentsScheduling and Executing Work on a NodeThe Head NodeDistributed Scheduling and ExecutionA Simple MapReduce Example with RayMapping and Shuffling Document DataReducing Word CountsSummary
Introducing Reinforcement LearningSetting Up a Simple Maze ProblemBuilding a SimulationTraining a Reinforcement Learning ModelBuilding a Distributed Ray AppRecapping RL TerminologySummary
An Overview of RLlibGetting Started with RLlibBuilding a Gym EnvironmentRunning the RLlib CLIUsing the RLlib Python APIConfiguring RLlib ExperimentsResource ConfigurationRollout Worker ConfigurationEnvironment ConfigurationWorking with RLlib EnvironmentsAn Overview of RLlib EnvironmentsWorking with Multiple AgentsWorking with Policy Servers and ClientsAdvanced ConceptsBuilding an Advanced EnvironmentApplying Curriculum LearningWorking with Offline DataOther Advanced TopicsSummary
Tuning HyperparametersBuilding a Random Search Example with RayWhy Is HPO Hard?An Introduction to TuneHow Does Tune Work?Configuring and Running TuneMachine Learning with TuneUsing RLlib with TuneTuning Keras ModelsSummary
Ray DatasetsRay Datasets BasicsComputing Over Ray DatasetsDataset PipelinesExample: Training Copies of a Classifier in ParallelExternal Library IntegrationsBuilding an ML PipelineSummary
The Basics of Distributed Model TrainingIntroduction to Ray Train by ExamplePredicting Big Tips in NYC Taxi RidesLoading, Preprocessing, and FeaturizationDefining a Deep Learning ModelDistributed Training with Ray TrainDistributed Batch InferenceMore on Trainers in Ray TrainMigrating to Ray Train with Minimal Code ChangesScaling Out TrainersPreprocessing with Ray TrainIntegrating Trainers with Ray TuneUsing Callbacks to Monitor TrainingSummary
Key Characteristics of Online InferenceML Models Are Compute IntensiveML Models Aren’t Useful in IsolationAn Introduction to Ray ServeArchitectural OverviewDefining a Basic HTTP EndpointScaling and Resource AllocationRequest BatchingMultimodel Inference GraphsEnd-to-End Example: Building an NLP-Powered APIFetching Content and PreprocessingNLP ModelsHTTP Handling and Driver LogicPutting It All TogetherSummary

Manually Creating a Ray ClusterDeployment on KubernetesSetting Up Your First KubeRay ClusterInteracting with the KubeRay ClusterExposing KubeRayConfiguring KubeRayConfiguring Logging for KubeRayUsing the Ray Cluster LauncherConfiguring Your Ray ClusterUsing the Cluster Launcher CLIInteracting with a Ray ClusterWorking with Cloud ClustersAWSUsing Other Cloud ProvidersAutoscalingSummary
Why Use AIR?Key AIR Concepts by ExampleRay Datasets and PreprocessorsTrainersTuners and CheckpointsBatch PredictorsDeploymentsWorkloads That Are Suited for AIRAIR Workload ExecutionAIR Memory ManagementAIR Failure ModelAutoscaling AIR WorkloadsSummary
A Growing EcosystemData Loading and ProcessingModel TrainingModel ServingBuilding Custom IntegrationsAn Overview of Ray’s IntegrationsRay and Other SystemsDistributed Python FrameworksRay AIR and the Broader ML EcosystemHow to Integrate AIR into Your ML PlatformWhere to Go from Here?Summary

Content preview from Learning Ray

Chapter 3. Building Your First Distributed Application

Now that you’ve seen the basics of the Ray API in action, let’s build something more realistic with it. By the end of this chapter, you will have built a reinforcement learning (RL) problem from scratch, implemented your first algorithm to tackle it, and used Ray tasks and actors to parallelize this solution to a local cluster—all in less than 250 lines of code.

This chapter is designed to work for readers who don’t have any experience with RL. We’ll work on a straightforward problem and develop the necessary skills to tackle it hands-on. Since Chapter 4 is devoted entirely to this topic, we’ll skip all advanced RL topics and language and just focus on the problem at hand. But even if you’re a quite advanced RL user, you’ll likely benefit from implementing a classic algorithm in a distributed setting.

This is the last chapter working only with Ray Core. We hope you learn to appreciate how powerful and flexible it is and how quickly you can implement distributed experiments that would otherwise take considerable efforts to scale.

Before we jump into any implementation, let’s quickly talk about the paradigm of RL in a bit more detail. Feel free to skip this section if you’ve worked with RL before.

Introducing Reinforcement Learning

One of my (Max’s) favorite mobile apps can automatically classify or “label” individual plants in our garden. The app works by simply showing it a picture of the plant in question. That’s immensely ...