Python Microservices Development

Task queues

The pattern used by Celery workers is a push-pull tasks queue. One service pushes messages into a specific queue, and some workers pick them up from the other end and perform an action on them. Each task goes to a single worker. Consider the following diagram:

There's no bidirectional communication. The sender just deposits a message in the queue and leaves. The next available worker gets the next message.

This blind, unidirectional message passing is perfect when you want to perform some asynchronous parallel tasks, and that makes it easy to scale.

Plus, once the sender has confirmed that the message was added in the broker, we ...