O'Reilly logo

Software Architecture with Python by Anand Balachandran Pillai

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Scaling workflows – message queues and task queues

One important aspect of scalability is to reducing coupling between systems. When two systems are tightly coupled, they prevent each other from scaling beyond a certain limit.

For example, a code written serially, where data and computation is tied into the same function, prevents the program from taking advantage of the existing resources like multiple CPU cores. When the same program is rewritten to use multiple threads (or processes) and a message passing system like a queue in between, we find it scales well to multiple CPUs. We've seen such examples aplenty in our concurrency discussion.

In a much similar way, systems over the Web scale better when they are decoupled. The classic example is ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required