Skip to Content
View all events

Distributed System Design Patterns

Published by O'Reilly Media, Inc.

Intermediate content levelIntermediate

Expert generalist foundations

What you’ll learn and how you can apply it

  • Gain an understanding of how distributed systems work
  • Break down team silos by understanding fundamental system trade-offs
  • Choose appropriate cloud services and use them effectively
  • Understand common distributed system patterns and build a foundational understanding of distributed system design

Course description

Cloud services like Amazon S3, Amazon EKS, and CosmosDB, along with products like Kafka, Kubernetes, Cassandra, Akka, or Blockchain, are all distributed systems that people use every day. Even if you’re not involved in building these kinds of systems, a deeper understanding can transform an engineer into an expert generalist—a practitioner who possesses “platform sympathy.”

Modern teams are often siloed into tool-specific specializations, focusing on product-specific trivia rather than foundational principles. An expert generalist breaks down these silos by understanding fundamental system trade-offs deeply enough to evaluate product internals, predict performance under saturation, and navigate architectural integration across messaging, database, control plane, and compute paradigms.

Unmesh Joshi uses hands-on code examples from Apache Kafka, Kubernetes, Delta Lake, and Apache Spark to demonstrate common distributed system patterns and help you build a foundational understanding of distributed system design.

This live event is for you because...

  • You’re an architect or developer who designs architecture for modern digital platforms.
  • You use various cloud services.
  • You build microservices architecture.

Prerequisites

  • JDK 21 installed on your computer
  • Golang
  • Your favorite editor
  • Familiarity with Java and Go (as the exercises will be in Java and Go)
  • Some experience with using message brokers like Kafka or a cloud service like Azure/AWS/GCP (helpful but not required)

Recommended follow-up:

Schedule

The time frames are only estimates and may vary according to how the class is progressing.

Day 1

Introduction (15 minutes)

  • Presentation: The patterns approach to understanding distributed systems

Group membership and failure detection (120 minutes)

  • Presentation: Using ZooKeeper, etcd, or a Raft cluster as KRaft for group membership; gossip protocols
  • Hands-on exercises: Implement membership tracking with ZooKeeper; implement a membership tracking with gossip protocols
  • Break

Partitioning patterns (120 minutes)

  • Presentation: Implementing a simple log-based storage; understanding basic partitioning techniques
  • Hands-on exercise: Enhance the log-based storage to use partitioning
  • Q&A

Day 2

Replication patterns (120 minutes)

  • Presentation: Understanding how to manage multiple replicas; leader and follower; high-watermark for log consistency
  • Hands-on exercise: Enhance the log storage with leader-based replication
  • Break

Control plane patterns (60 minutes)

  • Presentation: Understanding the need for cluster coordination
  • Hands-on exercise: Implement a controller for the replicated storage
  • Break

Distributed transactions (60 minutes)

  • Presentation: Understanding the need for transactions; implementing Delta-Lake style snapshot isolation-based transaction

Wrap-up and Q&A (15 minutes)

Your Instructor

Unmesh Joshi

Unmesh Joshi is a principal consultant at ThoughtWorks. He’s a software architecture enthusiast who believes that understanding the principles of distributed systems is as essential today as understanding web architecture or object-oriented programming was in the last decade.

linkedinXsearch

Skills covered

  • Distributed Systems
  • Serverless Architecture
  • Apache Spark
  • Kafka