Learning IPython for Interactive Computing and Data Visualization - Second Edition

Distributing tasks on several cores with IPython.parallel

In the previous sections, we covered a few methods to accelerate Python code. Here, we will see how to run multiple tasks in parallel on a multicore computer. IPython implements highly-powerful and user-friendly facilities for interactive parallel computing in the Notebook.

We first need to install ipyparallel (also called IPython.parallel) with conda install ipyparallel. Next, let's import NumPy and ipyparallel:

In [1]: import numpy as np
        # ipyparallel was IPython.parallel before IPython 4.0
        from ipyparallel import Client

To use IPython.parallel, we need to launch a few engines.

The first way to do it is to run ipcluster start in the terminal.

You can also launch engines from the Notebook ...