Real-World Event Sourcing

Preventing Duplicate Processing

When reading through the feature lists for message brokers and other event sourcing tools, you’ll often see two different phrases: at-least-once processing and at-most-once processing. If you’re lucky, you’ll even find systems (like NATS in some situations) that can guarantee exactly-once processing.

At-least-once processing means that you’re guaranteed to get the one event, but you could also theoretically receive multiple copies of the same event. Some applications and brokers deal with this problem by assigning unique IDs to events and using that ID to remove or ignore duplicates. In other products, you might see a “duplicate window,” where some configurable time exists during which the broker will purge messages ...