book

Python Concurrency with asyncio

by Matthew Fowler

February 2022

Intermediate to advanced

376 pages

10h 54m

English

Manning Publications

Read now

Unlock full access

prefaceacknowledgmentsabout this bookWho should read this book?How this book is organized: A road mapAbout the codeliveBook discussion forumabout the authorabout the cover illustration
1.1 What is asyncio?1.2 What is I/O-bound and what is CPU-bound?1.3 Understanding concurrency, parallelism, and multitasking1.3.1 Concurrency1.3.2 Parallelism1.3.3 The difference between concurrency and parallelism1.3.4 What is multitasking?1.3.5 The benefits of cooperative multitasking1.4 Understanding processes, threads, multithreading, and multiprocessing1.4.1 Process1.4.2 Thread1.5 Understanding the global interpreter lock1.5.1 Is the GIL ever released?1.5.2 asyncio and the GIL1.6 How single-threaded concurrency works1.6.1 What is a socket?1.7 How an event loop worksSummary
2.1 Introducing coroutines2.1.1 Creating coroutines with the async keyword2.1.2 Pausing execution with the await keyword2.2 Introducing long-running coroutines with sleep2.3 Running concurrently with tasks2.3.1 The basics of creating tasks2.3.2 Running multiple tasks concurrently2.4 Canceling tasks and setting timeouts2.4.1 Canceling tasks2.4.2 Setting a timeout and canceling with wait_for2.5 Tasks, coroutines, futures, and awaitables2.5.1 Introducing futures2.5.2 The relationship between futures, tasks, and coroutines2.6 Measuring coroutine execution time with decorators2.7 The pitfalls of coroutines and tasks2.7.1 Running CPU-bound code2.7.2 Running blocking APIs2.8 Accessing and manually managing the event loop2.8.1 Creating an event loop manually2.8.2 Accessing the event loop2.9 Using debug mode2.9.1 Using asyncio.run2.9.2 Using command-line arguments2.9.3 Using environment variablesSummary
3.1 Working with blocking sockets3.2 Connecting to a server with Telnet3.2.1 Reading and writing data to and from a socket3.2.2 Allowing multiple connections and the dangers of blocking3.3 Working with non-blocking sockets3.4 Using the selectors module to build a socket event loop3.5 An echo server on the asyncio event loop3.5.1 Event loop coroutines for sockets3.5.2 Designing an asyncio echo server3.5.3 Handling errors in tasks3.6 Shutting down gracefully3.6.1 Listening for signals3.6.2 Waiting for pending tasks to finishSummary
4.1 Introducing aiohttp4.2 Asynchronous context managers4.2.1 Making a web request with aiohttp4.2.2 Setting timeouts with aiohttp4.3 Running tasks concurrently, revisited4.4 Running requests concurrently with gather4.4.1 Handling exceptions with gather4.5 Processing requests as they complete4.5.1 Timeouts with as_completed4.6 Finer-grained control with wait4.6.1 Waiting for all tasks to complete4.6.2 Watching for exceptions4.6.3 Processing results as they complete4.6.4 Handling timeouts4.6.5 Why wrap everything in a task?Summary

5.1 Introducing asyncpg5.2 Connecting to a Postgres database5.3 Defining a database schema5.4 Executing queries with asyncpg5.5 Executing queries concurrently with connection pools5.5.1 Inserting random SKUs into the product database5.5.2 Creating a connection pool to run queries concurrently5.6 Managing transactions with asyncpg5.6.1 Nested transactions5.6.2 Manually managing transactions5.7 Asynchronous generators and streaming result sets5.7.1 Introducing asynchronous generators5.7.2 Using asynchronous generators with a streaming cursorSummary
6.1 Introducing the multiprocessing library6.2 Using process pools6.2.1 Using asynchronous results6.3 Using process pool executors with asyncio6.3.1 Introducing process pool executors6.3.2 Process pool executors with the asyncio event loop6.4 Solving a problem with MapReduce using asyncio6.4.1 A simple MapReduce example6.4.2 The Google Books Ngram dataset6.4.3 Mapping and reducing with asyncio6.5 Shared data and locks6.5.1 Sharing data and race conditions6.5.2 Synchronizing with locks6.5.3 Sharing data with process pools6.6 Multiple processes, multiple event loopsSummary
7.1 Introducing the threading module7.2 Using threads with asyncio7.2.1 Introducing the requests library7.2.2 Introducing thread pool executors7.2.3 Thread pool executors with asyncio7.2.4 Default executors7.3 Locks, shared data, and deadlocks7.3.1 Reentrant locks7.3.2 Deadlocks7.4 Event loops in separate threads7.4.1 Introducing Tkinter7.4.2 Building a responsive UI with asyncio and threads7.5 Using threads for CPU-bound work7.5.1 Multithreading with hashlib7.5.2 Multithreading with NumPySummary
8.1 Introducing streams8.2 Transports and protocols8.3 Stream readers and stream writers8.4 Non-blocking command-line input8.4.1 Terminal raw mode and the read coroutine8.5 Creating servers8.6 Creating a chat server and clientSummary
9.1 Creating a REST API with aiohttp9.1.1 What is REST?9.1.2 aiohttp server basics9.1.3 Connecting to a database and returning results9.1.4 Comparing aiohttp with Flask9.2 The asynchronous server gateway interface9.2.1 How does ASGI compare to WSGI?9.3 ASGI with Starlette9.3.1 A REST endpoint with Starlette9.3.2 WebSockets with Starlette9.4 Django asynchronous views9.4.1 Running blocking work in an asynchronous view9.4.2 Using async code in synchronous viewsSummary
10.1 Why microservices?10.1.1 Complexity of code10.1.2 Scalability10.1.3 Team and stack independence10.1.4 How can asyncio help?10.2 Introducing the backend-for-frontend pattern10.3 Implementing the product listing API10.3.1 User favorite service10.3.2 Implementing the base services10.3.3 Implementing the backend-for-frontend service10.3.4 Retrying failed requests10.3.5 The circuit breaker patternSummary
11.1 Understanding single-threaded concurrency bugs11.2 Locks11.3 Limiting concurrency with semaphores11.3.1 Bounded semaphores11.4 Notifying tasks with events11.5 ConditionsSummary
12.1 Asynchronous queue basics12.1.1 Queues in web applications12.1.2 A web crawler queue12.2 Priority queues12.3 LIFO queuesSummary
13.1 Creating a subprocess13.1.1 Controlling standard output13.1.2 Running subprocesses concurrently13.2 Communicating with subprocessesSummary
14.1 APIs with coroutines and functions14.2 Context variables14.3 Forcing an event loop iteration14.4 Using different event loop implementations14.5 Creating a custom event loop14.5.1 Coroutines and generators14.5.2 Generator-based coroutines are deprecated14.5.3 Custom awaitables14.5.4 Using sockets with futures14.5.5 A task implementation14.5.6 Implementing an event loop14.5.7 Implementing a server with a custom event loopSummary

Content preview from Python Concurrency with asyncio

Copyright

For online information and ordering of these and other Manning books, please visit www.manning.com. The publisher offers discounts on these books when ordered in quantity.

For more information, please contact

Special Sales Department

Manning Publications Co.

20 Baldwin Road

PO Box 761

Shelter Island, NY 11964

Email: orders@manning.com

No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher.

Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Start your free trial

Publisher Resources

ISBN: 9781617298660Supplemental Content Publisher Support Other Publisher Website Supplemental Content Purchase Link

Python Concurrency with asyncio

by Matthew Fowler

Copyright

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

You might also like

Python Testing with pytest

Python Testing with pytest

Hypermodern Python Tooling

Python for DevOps

Publisher Resources

Copyright

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,and much more.

You might also like

Python Testing with pytest

Python Testing with pytest

Hypermodern Python Tooling

Python for DevOps

Publisher Resources

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.