Twisted Network Programming Essentials, 2nd Edition

Chapter 10. Threads and Subprocesses

A mantra from Chapter 3 bears repeating: Twisted does not automatically make your code asynchronous or nonblocking.

What does Twisted do? It provides nonblocking primitives for common networking, filesystem, and timer activities, which wrap underlying nonblocking APIs exposed by the operating system. Twisted programs are event-driven; they use callbacks and are structured differently from synchronous programs. Twisted provides the Deferred abstraction to help manage these callbacks.

Even though Twisted programs use this event-driven model, sometimes you’ll still need to use threads or processes. This chapter covers some of the common cases and the relevant Twisted APIs.

Threads

In some cases—for example, when you’re using a blocking third-party API—the functions you’d like to use in your Twisted program aren’t under your control to be refactored into asynchronous ones using callbacks and Deferreds.

You are stuck with a blocking API, and you can’t use it as-is or you’ll block the entire event loop. To use it, you will need to make the blocking calls in threads. Twisted provides several methods related to making threaded calls, including:

callInThread

Execute a blocking function in its own thread.

deferToThread

Execute a blocking function in its own thread, and return the result as a Deferred.

In practice, deferToThread gets much more use than callInThread because you want a uniform interface to results, and Deferreds are that interface in Twisted programs. ...