book

Designing Event-Driven Systems

by Ben Stopford

May 2018

Intermediate to advanced

171 pages

3h 58m

English

O'Reilly Media, Inc.

Read now

Unlock full access

How to Read This BookAcknowledgments
Kafka Is Like REST but Asynchronous?Kafka Is Like a Service Bus?Kafka Is Like a Database?What Is Kafka Really? A Streaming Platform
The Log: An Efficient Structure for Retaining and Distributing MessagesLinear ScalabilitySegregating Load in Multiservice EcosystemsMaintaining Strong Ordering GuaranteesEnsuring Messages Are DurableLoad-Balance Services and Make Them Highly AvailableCompacted TopicsLong-Term Data StorageSecuritySummary
Commands, Events, and QueriesCoupling and Message BrokersIs Loose Coupling Always Good?Essential Data Coupling Is UnavoidableUsing Events for NotificationUsing Events to Provide State TransferWhich Approach to UseThe Event Collaboration PatternRelationship with Stream ProcessingMixing Request- and Event-Driven ProtocolsSummary
Making Services StatefulThe Event-Driven ApproachThe Pure (Stateless) Streaming ApproachThe Stateful Streaming ApproachThe Practicalities of Being StatefulSummary

Event Sourcing, Command Sourcing, and CQRS in a NutshellVersion Control for Your DataMaking Events the Source of TruthCommand Query Responsibility SegregationMaterialized ViewsPolyglot ViewsWhole Fact or Delta?Implementing Event Sourcing and CQRS with KafkaBuild In-Process Views with Tables and State Stores in Kafka StreamsWriting Through a Database into a Kafka Topic with Kafka ConnectWriting Through a State Store to a Kafka Topic in Kafka StreamsUnlocking Legacy Systems with CDCQuery a Read-Optimized View Created in a DatabaseMemory Images/Prepopulated CachesThe Event-Sourced ViewSummary
Encapsulation Isn’t Always Your FriendThe Data DichotomyWhat Happens to Systems as They Evolve?The God Service ProblemThe REST-to-ETL ProblemMake Data on the Outside a First-Class CitizenDon’t Be Afraid to EvolveSummary
A Database Inside OutSummary
If Messaging Remembers, Databases Don’t Have ToTake Only the Data You Need, Nothing MoreRebuilding Event-Sourced ViewsKafka StreamsDatabases and CachesHandling the Impracticalities of Data MovementAutomation and Schema MigrationThe Data Divergence ProblemSummary
Eventual ConsistencyTimelinessCollisions and MergingThe Single Writer PrincipleCommand TopicSingle Writer Per TransitionAtomicity with TransactionsIdentity and Concurrency ControlLimitationsSummary
The Duplicates ProblemUsing the Transactions API to Remove DuplicatesExactly Once Is Both Idempotence and Atomic CommitHow Kafka’s Transactions Work Under the CoversStore State and Send Events AtomicallyDo We Need Transactions? Can We Do All This with Idempotence?What Can’t Transactions Do?Making Use of Transactions in Your ServicesSummary
Using Schemas to Manage the Evolution of Data in TimeHandling Schema Change and Breaking Backward CompatibilityCollaborating over Schema ChangeHandling Unreadable MessagesDeleting DataTriggering Downstream DeletesSegregating Public and Private TopicsSummary
A Simple Email Service Built with Kafka Streams and KSQLWindows, Joins, Tables, and State StoresSummary
An Order Validation EcosystemJoin-Filter-ProcessEvent-Sourced Views in Kafka StreamsCollapsing CQRS with a Blocking ReadScaling Concurrent Operations in Streaming SystemsRekey to JoinRepartitioning and Staged ExecutionWaiting for N EventsReflecting on the DesignA More Holistic Streaming EcosystemSummary