System Design on AWS

Chapter 12. AWS Messaging, Orchestration, Monitoring, and Access Management Services

In systems running at a large scale, multiple components interact and coordinate with one another to perform tasks, which is called orchestration. This communication and coordination among application components can be managed by different AWS services, and we can pick and choose our services based on our use case. In this chapter, we’ll deep dive into core orchestration services like Amazon Simple Notification Service (SNS), which can be used to broadcast messages to multiple subscribers, and AWS Step Functions, which can be used to coordinate and execute different AWS or custom services in a specific order.

We’ll begin with a discussion of AWS services that help us achieve the pub/sub design pattern in the AWS cloud environment, such as Amazon Managed Streaming for Apache Kafka (MSK), Amazon Kinesis, Amazon Simple Queue Service (SQS), and Amazon SNS. Then, we’ll move on to monitoring, authorization, and authentication services, such as Amazon CloudWatch, AWS IAM, and Amazon Cognito.

Amazon Managed Streaming for Apache Kafka

We discussed pub/sub architecture, message brokers, and message queues in Chapter 8; this chapter introduces several services that AWS offers for designing applications on top of these concepts. Apache Kafka is an open source message broker and event-streaming platform used extensively for designing event-driven architectures. AWS offers a managed service for Apache Kafka, ...