Cloud Native Development Patterns and Best Practices

Resulting context

The primary benefit of this solution is that we are not relying on a self-managed streaming cluster to retain a history of all events and double as a data lake. Managing this level of disk storage takes a great deal of effort and can expose a system to the risk of significant data loss if not managed properly. Instead, this solution enables teams to leverage value-added cloud-streaming services so that they can focus on the functional requirements of their components. The data lake is responsible for the long-term durable storage of all the events, while the streams run lean and just retain the most recent events. This ultimately helps ensure that we have proper bulkheads for the streams, instead of the tendency to have ...