Skip to Content
View all events

Event-Driven Architecture and Data

Published by O'Reilly Media, Inc.

Intermediate content levelIntermediate

A hands-on course for extracting and using events from existing data stores

Getting started with event-driven architectures requires access to quality event data. In a typical organization, most of this data is encapsulated within monolithic data stores and must be liberated into event streams to be of use for the rest of the organization. Extracting and transforming that important business domain data into events is a fundamental part of moving to event-driven architectures.

Expert Adam Bellemare takes you through data extraction techniques such as change-data capture, native production, and outbox tables. You’ll learn about the complexities introduced by distributed time, the need for determinism, failures, and late-arriving events as you follow a hypothetical organization’s journey from a single monolith to a distributed event-driven data communication layer. Join in to explore the trade-offs and decisions this organization has to make along the way—they might just help you come up with plans that work for your own company.

What you’ll learn and how you can apply it

By the end of this live online course, you’ll understand:

  • The typical journey that a company may undertake when moving to event-driven architectures
  • How to use change-data capture, native production, and the outbox pattern to extract data from data stores
  • The performance, coupling, and maintenance trade-offs of each extraction mechanism
  • How to structure and model events based on extracted data
  • The downstream implications of the various extraction mechanisms and how to mitigate and resolve some of the difficulties you’ll face

And you’ll be able to:

  • Plan your own organization’s journey to well-formed event-streams based on core business data
  • Identify the best mechanisms for extracting important business data into event streams specific to your organization
  • Determine where you are in this journey and the upcoming hurdles you’ll face next
  • Plan the evolution of your data models and event-sourcing strategies

This live event is for you because...

  • You’re an intermediate to advanced software developer and system architecture professional or data engineer.
  • You work with business strategies, bounded contexts, and infrastructural prerequisites.
  • You want to learn how to migrate your organization to an event-driven microservice architecture.

Prerequisites

  • A basic understanding of a publish-subscribe event broker such as Apache Kafka (Message brokers such as RabbitMQ and ActiveMQ aren’t suitable for the purposes of forming the universal data communication layer required for event-driven architectures.)
  • Familiarity with bounded contexts, domain-driven design, and microservice architectures (useful but not required)

Recommended preparation:

Recommended follow-up:

Schedule

The time frames are only estimates and may vary according to how the class is progressing.

Introduction (10 minutes)

  • Presentation and demo: The typical journey of an organization moving into event-driven architectures; the need for well-structured events; bootstrapping existing data sources—relational tables and other sources of domain data; forked writes; CDC; the outbox pattern; native production; goals; trade-offs; design

Change-data capture pattern (25 minutes)

  • Presentation and demo: Overview of the change-data capture pattern; why you’d use it; benefits and trade-offs; practical application; data deltas—before and after; the resultant event streams and how to use them

Time and determinism (20 minutes)

  • Presentation and demo: Event time, wall-clock time, and query time; How does time affect our stream consumers?; determinism
  • Q&A

Break (5 minutes)

Event design (25 minutes)

  • Presentation and demo: How do you design events?; current state versus differences a.k.a. noun versus verb;changing events through time

Outbox tables capture pattern (30 minutes)

  • Presentation and demo: Overview of the outbox tables capture pattern; why you’d use it; native production versus change-data capture of outbox; benefits and trade-offs; practical application; impact on both producers and consumers Q&A

Break (5 minutes)

Using relational events: Streaming joins of materialized datasets (10 minutes)

  • Presentation and demo: Stream-table joins and table-table joins; primary key joins; foreign key joins; difficulties and implications for consumers

Eventification and event-stream scaling (10 minutes)

  • Presentation and demo: Number of event streams; frequency of updates; relationships between events; reducing the burden of joins on downstream consumers; principles

Access and data discovery (10 minutes)

  • Presentation and demo: Principles; identities and access control lists; namespacing and private streams

As time goes on, what to expect when using event streams (30 minutes)

  • Presentation and demo: Increasing volume of events; increasing state requirements; changing partition parallelism; modifying the event domain; breaking schema changes; system and stream deprecation; stream migrations
  • Q&A

Your Instructor

Adam Bellemare

Adam Bellemare is a principal technologist at Confluent. He’s also the author of Building Event-Driven Microservices and Building an Event-Driven Data Mesh, both published by O'Reilly. Previously, he worked in data platform engineering at Shopify, Flipp, and BlackBerry.

Skill covered

Software Architecture