ZeroMQ

Designing Reliability

So, to make things brutally simple, reliability is “keeping things working properly when code freezes or crashes,” a situation we’ll shorten to “when code dies.” However, the things we want to keep working properly are more complex than just messages. We need to take each core ØMQ messaging pattern and see how to make it work (if we can) even when code dies.

Let’s take them one by one:

Request-reply:

If the server dies while processing a request, the client can figure that out because it won’t get an answer back. Then it can give up in a huff, wait and try again later, find another server, etc. As for the client dying, we can brush that off as “someone else’s problem” for now.

Publish-subscribe

If the client dies (having gotten some data), the server won’t know about it. Pub-sub doesn’t send any information back from the client to the server. However, the client can contact the server out-of-band—e.g., via request-reply—and say, “Please resend everything I missed.” As for the server dying, that’s outside the scope of this discussion. Subscribers can also self-verify that they’re not running too slowly, and take action (e.g., warn the operator and die) if they are.

Pipeline

If a worker dies (while working), the ventilator doesn’t know about it. Pipelines, like pub-sub and the grinding gears of time, only work in one direction. But the downstream collector can detect that one task didn’t get done, and send a message back to the ventilator saying, “Hey, resend task 324!” ...