Learn AWS Serverless Computing

Micro-batch

The use cases for micro-batch data processing can be similar to traditional batch processing. The main differences are that the data will be processed more often and in smaller volumes. When designing a micro-batch process, there are two ways to approach the scheduling model. It can be based on the following:

• Time interval-based: This is when you process data on a predefined schedule, for example, every 5 minutes.
• Event-based: This is when you choose to process data when the amount of data that's available or queued for processing reaches a defined threshold, for example, every 2 MB or every 1,000 records.

Apache Spark is one of the most popular open source tools for micro-batch workloads—AWS Glue is also based on Apache Spark. ...