Fundamentals of Event-Driven Microservices
Building architecture to support your organization’s data
Topic: System Administration
An organization is only as good as its data, and enabling access to that data is a critical part of how any system delivers value. Conventional system architectures may not be up to the task, but event-driven microservices can allow large-scale data usage across the business units in your organization.
Join expert Adam Bellemare to rethink how your data works in modern day communication streams. You’ll learn how to unleash the power of your data in an event-driven architecture with an event broker as the single source of truth, streamlining and standardizing communication and access to business-critical data across your organization. And you’ll understand how event-driven microservices complement the data communication layer provided by the event broker, allowing you to harness these streams to unlock real business value.
What you'll learn-and how you can apply it
By the end of this live online course, you’ll understand:
- The difficulties that modern organizations face with regards to their data, particularly around access and consistency
- The components involved in an event-driven architecture, including the role of the event broker as the single source of truth
- Best practices for bootstrapping existing data sources into event streams and principles for creating new event streams
- How microservices are a natural processing extension of universal data-access through the event broker
And you’ll be able to:
- Identify the trade-offs of an event-driven architecture and align them with the needs of your organization
- Identify the common entities and events that are the most valuable for your organization and provision them as event streams from your event broker
- Build basic stateless and stateful microservices
This training course is for you because...
- You’re an intermediate to advanced software developer and system architecture professional or data engineer.
- You work with business strategies, bounded contexts, and infrastructural prerequisites.
- You want to learn how to migrate your organization to an event-driven microservice architecture.
- A basic understanding of a publish-subscribe event broker such as Apache Kafka (Message brokers such as RabbitMQ and ActiveMQ aren’t suitable for the purposes of forming the universal data communication layer required for event-driven architectures.)
- Familiarity with bounded contexts, domain-driven design, and microservice architectures (useful but not required)
- Read the introduction to Kafka (supporting documentation)
- Read Designing Event-Driven Systems (book)
- Read “Managing Complexity with Bounded Contexts” (chapter 3 in What Is Domain-Driven Design?)
About your instructor
Adam Bellemare is a Staff Engineer, Data Platform at Shopify. Previously, he was at Flipp from 2014, first as a Senior Developer, followed by a role as Staff. He has also held positions in embedded software development and quality assurance. His expertise includes: Devops (Kafka, Spark, Mesos, Zookeeper Clusters. Programmatic Building, scaling, destroying); Technical Leadership (Bringing Avro formatting to our data end-to-end, championing Kafka as the event-driven microservice bus, prototyping JRuby, Scala and Java Kafka clients and focusing on removing technical impediments to allow for product delivery); Software Development (Building microservices in Java and Scala using Spark and Kafka libraries); and Data Engineering (Reshaping the way that behavioral data is collected from user devices and shared with our Machine Learning, Billing and Analytics teams).
The timeframes are only estimates and may vary according to how the class is progressing
Difficulties in modern day data communication (10 minutes)
- Presentation: Commonly used solutions and their problems (scale, freshness, coupling, fan-out); how tools shape your perceptions of what is and isn’t okay
- Group discussion: Are there any alternatives that we didn’t cover?
Data communication layer and event streams (10 minutes)
- Presentation: Foundations, single source of truth, responsibilities of publishers, and consumers; the demands and the tools—why this works today when it didn't before
The primitives of the data communication layer (10 minutes)
- Presentation: Streams, event types, materialization, and communications state
- Helpful conventions: Why and what they're for (5 minutes)
- Presentation: Single source of truth; single writer principle; single schema per stream; access control
Break (10 minutes)
Building the DCL (25 minutes)
- Presentation: Requirements—schema registry support, indefinite retention, and tiered storage; part 1 of the microservice tax; comparing services—open source, hosted, mass adoption, and the de facto standard in Apache Kafka
Organizational and cultural shifts in your organization (10 minutes)
- Presentation: Where most organizations start out; where we want to go
Bootstrapping data into the data communication layer (20 minutes)
- Presentation: Strategies, techniques, and trade-offs
Break (10 minutes)
Defining microservices (10 minutes)
- Presentation: Bounded contexts and domain-driven design; managing microservices—containers, compute resources, deployment, logging, and authentication
Using the data communication layer (25 minutes)
- Presentation: Building microservices using stream-based composition; materializing state; internal state, external state, and sharing state
Determinism (10 minutes)
- Presentation: Time stamps, ordering, and late events in brief
Coordinating multiple services (10 minutes)
- Presentation: Orchestration and choreography
Wrap-up and Q&A (15 minutes)