System Design on AWS

Chapter 10. AWS Orchestration Services

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 12th chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at [e-mail].

For running systems at large scale, multiple components interact and coordinate with each other to perform any task referred to as orchestration. This communication and coordination between application components can be managed by different AWS services and we can pick and choose our services based on our use-case. In this chapter, we’ll deep dive into core orchestration services such as Simple Notification Service (SNS) which can be used to broadcast a message to multiple subscribers and AWS Step Functions which can be used for coordination and execution of different AWS or custom services in specific order.

We’ll start off with discussion around AWS services which are helpful in achieving publisher-subscriber design pattern in the AWS Cloud environment such as Simple Queue Service (SQS), SNS & Amazon Managed Streaming for Apache Kafka (MSK) and then later on move on to monitoring, authorization ...