Distributed Computing with Python

Closing thoughts

One of the main difficulties in developing parallel applications is getting data access right and avoiding race conditions or situations that would corrupt shared data. Sometimes, these situations are easy to spot as they lead to spectacular crashes. Other times, more worryingly, they are not—the application keeps plodding along, producing incorrect results.

It is always important to have good tests for our applications and their internal functions. It is even more so for parallel applications, where building a clear mental picture of what happens where and when can be particularly challenging.

Another difficulty in parallelizing algorithms is to know when to stop. Amdahl's law tells us very clearly that parallelization is, from ...