Scaling Python with Ray

Chapter 2. Getting Started with Ray (Locally)

As we’ve discussed, Ray is useful for managing resources from a single computer up to a cluster. It is simpler to get started with a local installation, which leverages the parallelism of multicore/multi-CPU machines. Even when deploying to a cluster, you’ll want to have Ray installed locally for development. Once you’ve installed Ray, we’ll show you how to make and call your first asynchronous parallelized function and store state in an actor.

Tip

If you are in a hurry, you can also use Gitpod on the book’s GitHub repo to get a web environment with the examples, or check out Anyscale’s managed Ray.

Installation

Installing Ray, even on a single machine, can range from relatively straightforward to fairly complicated. Ray publishes wheels to the Python Package Index (PyPI) following a normal release cadence as well as in nightly releases. These wheels are currently available for only x86 users, so ARM users will mostly need to build Ray from source.¹

Tip

M1 ARM users on macOS can use the x86 packages with Rosetta. Some performance degradation occurs, but it’s a much simpler setup. To use the x86s package, install Anaconda for macOS.

Installing for x86 and M1 ARM

Most users can run pip install -U ray to automatically install Ray from PyPI. When you go to distribute your computation on multiple machines, it’s often easier to have been working in a Conda environment so you can match Python versions with your cluster and know your package ...