Skip to Content
Functional Python Programming - Second Edition
book

Functional Python Programming - Second Edition

by Steven F. Lott
April 2018
Intermediate to advanced content levelIntermediate to advanced
408 pages
10h 42m
English
Packt Publishing
Content preview from Functional Python Programming - Second Edition

Using the concurrent.futures module

In addition to the multiprocessing package, we can also make use of the concurrent.futures module. This also provides a way to map data to a concurrent pool of threads or processes. The module API is relatively simple and similar in many ways to the multiprocessing.Pool() function's interface.

Here is an example to show just how similar they are:

from concurrent.futures import ProcessPoolExecutor
pool_size = 4
pattern = "*.gz"
combined = Counter()
with ProcessPoolExecutor(max_workers=pool_size) as workers:
    for result in workers.map(analysis, glob.glob(pattern)):
        combined.update(result) 

The most significant change between the preceding example and previous examples is that we're using an instance of the ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Functional Python Programming - Third Edition

Functional Python Programming - Third Edition

Steven F. Lott

Publisher Resources

ISBN: 9781788627061Supplemental Content