Skip to Content
View all events

Data Design for Microservices

Published by O'Reilly Media, Inc.

Intermediate content levelIntermediate

How to handle data in an increasingly distributed world

Course outcomes

  • Understand how a microservice architecture changes how data is stored and accessed
  • Acquire tips and tricks to decompose data from monolithic databases
  • Process data from multiple microservices for reporting purposes
  • Understand the changes needed to help shift from batch to real-time data processing

A microservice architecture focuses on the property of independent deployability, which allows teams to work with more autonomy and get software into the hands of users quickly. But this requires thinking differently about data: Instead of a single, giant database, data must be decomposed to allow each microservice to be changed independently.

Join expert Sam Newman to learn more about the change in how data is stored and accessed in a microservices world. You’ll see how to address practical concerns like referential integrity and analytics, and you’ll learn how microservices are helping shift the thinking around analytical data from batch-oriented to real-time processing.

What you’ll learn and how you can apply it

  • Understand how data is stored and accessed in a microservices world in order to work with these architectures effectively

This live event is for you because...

  • You’re a developer, architect, data scientist, or data engineer.
  • You work with a monolithic system and want to understand the implications of changing to a microservice architecture.
  • You’re moving from a monolith to microservice architecture and want to understand what will need to change around data.
  • You already have a microservice architecture and are struggling to work with data in the way you want.

Prerequisites

  • A high-level understanding of microservices (practical experience with microservices is not required)

Recommended preparation:

Recommended follow-up:

Schedule

The time frames are only estimates and may vary according to how the class is progressing.

Microservices and data (15 minutes)

  • Presentation: An extremely brief introduction to microservices; why data is handled differently (information hiding)

Operational versus analytical data (15 minutes)

  • Presentation: The differences between OLTP and OLAP; how different use cases want to use data in different ways—competing concerns; microservices and data are really about OLTP—so what about OLAP needs?

Problems with analytical data in a microservices world (25 minutes)

  • Presentation: Traditional access methods breaking information hiding; data pipelines constraining changes to microservices; the analytical data user as second-class citizen; canonical data dictionaries versus hidden data
  • Break

Data consumers as first-class citizens (15 minutes)

  • Presentation: Treating data consumers the same as other consumers; mixing endpoints to deliver different needs; treating data as a product

Reporting database patterns (10 minutes)

  • Presentation: Implementation options—views, CDC

Analytical data store patterns (15 minutes)

  • Presentation: Data warehouses; data lakes; data lakehouses

Real-time versus batch (30 minutes)

  • Presentation: A new vision for what “analytical” means; bulk publishing to streaming; Kafka for data streaming; stream processing frameworks (Spark, Flink)
  • Break

Data mesh (15 minutes)

  • Presentation: What is a service mesh?

Migrating monolithic databases overview (10 minutes)

Static reference data example (20 minutes)

  • Presentation: Options for static reference data in microservices

Data relationships in microservices (20 minutes)

  • Presentation: Foreign keys versus cross-service references; ID schemes; referential integrity; cascading deletes Break

Referential integrity and microservices (30 minutes)

  • Presentation: Soft deletes; data duplication; event sourcing; distributed transactions

Cascading deletes and microservices (10 minutes)

Wrap-up (10 minutes)

Your Instructor

Sam Newman

Sam Newman is a technologist focusing on the areas of cloud, microservices, and continuous delivery—three topics which seem to overlap frequently. He provides consulting, training, and advisory services to startups and large multinational enterprises alike, drawing on his more than 20 years in IT as a developer, sysadmin, and architect. Sam is the author of the best-selling Building Microservices and Monolith to Microservices, both from O’Reilly, and is also an experienced conference speaker.

Skill covered

Microservices