book

Scaling Python with Ray

by Holden Karau, Boris Lublinsky

November 2022

Intermediate to advanced

266 pages

6h 14m

English

O'Reilly Media, Inc.

Read now

Unlock full access

What You Will LearnA Note on ResponsibilityConventions Used in This BookLicenseUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgmentsFrom HoldenFrom Boris
Why Do You Need Ray?Where Can You Run Ray?Running Your Code with RayWhere Does It Fit in the Ecosystem?Big Data / Scalable DataFramesMachine LearningWorkflow SchedulingStreamingInteractiveWhat Ray Is NotConclusion
InstallationInstalling for x86 and M1 ARMInstalling (from Source) for ARMHello WorldsRay Remote (Task/Futures) Hello WorldData Hello WorldActor Hello WorldConclusion
Essentials of Ray Remote FunctionsComposition of Remote Ray FunctionsRay Remote Best PracticesBringing It Together with an ExampleConclusion
Understanding the Actor ModelCreating a Basic Ray Remote ActorImplementing the Actor’s PersistenceScaling Ray Remote ActorsRay Remote Actors Best PracticesConclusion
Fault ToleranceRay ObjectsSerialization/PicklingcloudpickleApache ArrowResources / Vertical ScalingAutoscalerPlacement Groups: Organizing Your Tasks and ActorsNamespacesManaging Dependencies with Runtime EnvironmentsDeploying Ray Applications with the Ray Job APIConclusion
Apache KafkaBasic Kafka ConceptsKafka APIsUsing Kafka with RayScaling Our ImplementationBuilding Stream-Processing Applications with RayKey-Based ApproachKey-Independent ApproachGoing Beyond KafkaConclusion
Understanding Microservice Architecture in RayDeploymentAdditional Deployment CapabilitiesDeployment CompositionUsing Ray Serve for Model ServingSimple Model Service ExampleConsiderations for Model-Serving ImplementationsSpeculative Model Serving Using the Ray Microservice FrameworkConclusion
What Is Ray Workflows?How Is It Different from Other Solutions?Ray Workflows FeaturesWhat Are the Main Features?Workflow PrimitivesWorking with Basic Workflow ConceptsWorkflows, Steps, and ObjectsDynamic WorkflowsVirtual ActorsWorkflows in Real LifeBuilding WorkflowsManaging WorkflowsBuilding a Dynamic WorkflowBuilding Workflows with Conditional StepsHandling ExceptionsHandling Durability GuaranteesExtending Dynamic Workflows with Virtual ActorsIntegrating Workflows with Other Ray PrimitivesTriggering Workflows (Connecting to Events)Working with Workflow MetadataConclusion

Creating and Saving Ray DatasetsUsing Ray Datasets with Different ToolsUsing Tools on Ray Datasetspandas-like DataFrames with DaskIndexingShufflesEmbarrassingly Parallel OperationsWorking with Multiple DataFramesWhat Does Not WorkWhat’s SlowerHandling Recursive AlgorithmsWhat Other Functions Are Differentpandas-like DataFrames with ModinBig Data with SparkWorking with Local ToolsUsing Built-in Ray Dataset OperationsImplementing Ray DatasetsConclusion
Using scikit-learn with RayUsing Boosting Algorithms with RayUsing XGBoostUsing LightGBMUsing PyTorch with RayReinforcement Learning with RayHyperparameter Tuning with RayConclusion
What Are GPUs Good At?The Building BlocksHigher-Level LibrariesAcquiring and Releasing GPU and Accelerator ResourcesRay’s ML LibrariesAutoscaler with GPUs and AcceleratorsCPU Fallback as a Design PatternOther (Non-GPU) AcceleratorsConclusion
Ray Dependency Security IssuesInteracting with the Existing ToolsUsing Ray with CI/CD ToolsAuthentication with RayMultitenancy on RayCredentials for Data SourcesPermanent Versus Ephemeral ClustersEphemeral ClustersPermanent ClustersMonitoringInstrumenting Your Code with Ray MetricsWrapping Custom Programs with RayConclusion
High-Level DesignImplementationOutbound Mail ClientShared Actor Patterns and UtilitiesMail Server ActorSatellite ActorUser ActorSMS Actor and Serve ImplementationTestingDeploymentConclusion
Installing Ray LocallyUsing Ray Docker ImagesUsing Ray ClustersInstalling Ray on AWSInstalling Ray on IBM CloudInstalling Ray on KubernetesInstalling Ray on a kind ClusterUsing ray upUsing the Ray Kubernetes OperatorInstalling Ray on OpenShiftConclusion
General Debugging Tips with RaySerialization ErrorsLocal Debugging with Ray LocalRemote DebuggingRay’s Integrated Debugger (via Pdb)Other ToolsRay and Container Exit CodesRay LogsContainer ErrorsNative ErrorsConclusion

Overview

Serverless computing enables developers to concentrate solely on their applications rather than worry about where they've been deployed. With the Ray general-purpose serverless implementation in Python, programmers and data scientists can hide servers, implement stateful applications, support direct communication between tasks, and access hardware accelerators.

In this book, experienced software architecture practitioners Holden Karau and Boris Lublinsky show you how to scale existing Python applications and pipelines, allowing you to stay in the Python ecosystem while reducing single points of failure and manual scheduling. Scaling Python with Ray is ideal for software architects and developers eager to explore successful case studies and learn more about decision and measurement effectiveness.

If your data processing or server application has grown beyond what a single computer can handle, this book is for you. You'll explore distributed processing (the pure Python implementation of serverless) and learn how to: