Hands-On Design Patterns and Best Practices with Julia

Understanding the overhead of parallel processing

Have you noticed something interesting here? Since we have 16 worker processes, we would have expected that the parallel processing function to be close to 16 times faster. But the result came in at around 6 times, which is somewhat less than we expected. Why is that?

The answer is that it is just a matter of scale. There is some performance overhead to use the parallel processing facility. Typically, this overhead can be ignored because it is immaterial when compared to the amount of work being performed. In this particular example, calculating standard deviation is a really trivial computation. So, in relative terms, the overhead of coordinating remote function calls and collecting results ...