book

Scaling Python with Ray

by Holden Karau, Boris Lublinsky

November 2022

Intermediate to advanced

266 pages

6h 14m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Foreword
Preface
What You Will LearnA Note on ResponsibilityConventions Used in This BookLicenseUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgmentsFrom HoldenFrom Boris
1. What Is Ray, and Where Does It Fit?
Why Do You Need Ray?Where Can You Run Ray?Running Your Code with RayWhere Does It Fit in the Ecosystem?Big Data / Scalable DataFramesMachine LearningWorkflow SchedulingStreamingInteractiveWhat Ray Is NotConclusion
2. Getting Started with Ray (Locally)
InstallationInstalling for x86 and M1 ARMInstalling (from Source) for ARMHello WorldsRay Remote (Task/Futures) Hello WorldData Hello WorldActor Hello WorldConclusion
3. Remote Functions
Essentials of Ray Remote FunctionsComposition of Remote Ray FunctionsRay Remote Best PracticesBringing It Together with an ExampleConclusion
4. Remote Actors
Understanding the Actor ModelCreating a Basic Ray Remote ActorImplementing the Actor’s PersistenceScaling Ray Remote ActorsRay Remote Actors Best PracticesConclusion
5. Ray Design Details
Fault ToleranceRay ObjectsSerialization/PicklingcloudpickleApache ArrowResources / Vertical ScalingAutoscalerPlacement Groups: Organizing Your Tasks and ActorsNamespacesManaging Dependencies with Runtime EnvironmentsDeploying Ray Applications with the Ray Job APIConclusion
6. Implementing Streaming Applications
Apache KafkaBasic Kafka ConceptsKafka APIsUsing Kafka with RayScaling Our ImplementationBuilding Stream-Processing Applications with RayKey-Based ApproachKey-Independent ApproachGoing Beyond KafkaConclusion
7. Implementing Microservices
Understanding Microservice Architecture in RayDeploymentAdditional Deployment CapabilitiesDeployment CompositionUsing Ray Serve for Model ServingSimple Model Service ExampleConsiderations for Model-Serving ImplementationsSpeculative Model Serving Using the Ray Microservice FrameworkConclusion
8. Ray Workflows
What Is Ray Workflows?How Is It Different from Other Solutions?Ray Workflows FeaturesWhat Are the Main Features?Workflow PrimitivesWorking with Basic Workflow ConceptsWorkflows, Steps, and ObjectsDynamic WorkflowsVirtual ActorsWorkflows in Real LifeBuilding WorkflowsManaging WorkflowsBuilding a Dynamic WorkflowBuilding Workflows with Conditional StepsHandling ExceptionsHandling Durability GuaranteesExtending Dynamic Workflows with Virtual ActorsIntegrating Workflows with Other Ray PrimitivesTriggering Workflows (Connecting to Events)Working with Workflow MetadataConclusion

9. Advanced Data with Ray
Creating and Saving Ray DatasetsUsing Ray Datasets with Different ToolsUsing Tools on Ray Datasetspandas-like DataFrames with DaskIndexingShufflesEmbarrassingly Parallel OperationsWorking with Multiple DataFramesWhat Does Not WorkWhat’s SlowerHandling Recursive AlgorithmsWhat Other Functions Are Differentpandas-like DataFrames with ModinBig Data with SparkWorking with Local ToolsUsing Built-in Ray Dataset OperationsImplementing Ray DatasetsConclusion
10. How Ray Powers Machine Learning
Using scikit-learn with RayUsing Boosting Algorithms with RayUsing XGBoostUsing LightGBMUsing PyTorch with RayReinforcement Learning with RayHyperparameter Tuning with RayConclusion
11. Using GPUs and Accelerators with Ray
What Are GPUs Good At?The Building BlocksHigher-Level LibrariesAcquiring and Releasing GPU and Accelerator ResourcesRay’s ML LibrariesAutoscaler with GPUs and AcceleratorsCPU Fallback as a Design PatternOther (Non-GPU) AcceleratorsConclusion
12. Ray in the Enterprise
Ray Dependency Security IssuesInteracting with the Existing ToolsUsing Ray with CI/CD ToolsAuthentication with RayMultitenancy on RayCredentials for Data SourcesPermanent Versus Ephemeral ClustersEphemeral ClustersPermanent ClustersMonitoringInstrumenting Your Code with Ray MetricsWrapping Custom Programs with RayConclusion
A. Space Beaver Case Study: Actors, Kubernetes, and More
High-Level DesignImplementationOutbound Mail ClientShared Actor Patterns and UtilitiesMail Server ActorSatellite ActorUser ActorSMS Actor and Serve ImplementationTestingDeploymentConclusion
B. Installing and Deploying Ray
Installing Ray LocallyUsing Ray Docker ImagesUsing Ray ClustersInstalling Ray on AWSInstalling Ray on IBM CloudInstalling Ray on KubernetesInstalling Ray on a kind ClusterUsing ray upUsing the Ray Kubernetes OperatorInstalling Ray on OpenShiftConclusion
C. Debugging with Ray
General Debugging Tips with RaySerialization ErrorsLocal Debugging with Ray LocalRemote DebuggingRay’s Integrated Debugger (via Pdb)Other ToolsRay and Container Exit CodesRay LogsContainer ErrorsNative ErrorsConclusion
Index
About the Authors

Content preview from Scaling Python with Ray

About the Authors

Holden Karau is a queer transgender Canadian, Apache Spark committer, Apache Software Foundation member, and an active open source contributor. As a software engineer, she’s worked on a variety of distributed computing, search, and classification problems at Apple, Google, IBM, Alpine, Databricks, Foursquare, and Amazon. She graduated from the University of Waterloo with a bachelor of mathematics in computer science. Outside of software, she enjoys playing with fire, welding, riding scooters, eating poutine, and dancing.

Boris Lublinsky is a chief architect for IBM’s Discovery Accelerator Platform, where he specializes in Kubernetes, serverless, workflows, and complex systems design. Boris has over 30 years of experience in enterprise architecture and software development. He is the coauthor of Applied SOA (Wiley), Professional Hadoop Solutions (Wiley), Serving Machine Learning Models (O’Reilly), and Kubeflow for Machine Learning (O’Reilly). He is also a contributor to several open source projects. Boris is a frequent speaker at numerous industry conferences and cofounder of several Chicago user groups.

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Interpretable Machine Learning with Python - Second Edition

Publisher Resources

ISBN: 9781098118792Errata Page

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Scaling Python with Ray

by Holden Karau, Boris Lublinsky

About the Authors

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.