Python High Performance - Second Edition

Dask distributed

The first iterations of the Dask project were designed to run on a single computer using a thread-based or a process-based scheduler. Recently, the implementation of a new distributed backend can be used to set up and run Dask graphs on a network of computers.

Dask distributed is not installed automatically with Dask. The library is available through the conda package manager (use the $ conda install distributed command ) as well as pip (with the $ pip install distributed command).

Getting started with Dask distributed is really easy. The most basic setup is obtained by instantiating a Client object:

    from dask.distributed import Client         client = Client()    # Result: # <Client: scheduler='tcp://127.0.0.1:46472' processes=4 ...