The simplest form of batch processing is a work queue. In a work queue
system, there is a batch of work to be performed. Each piece of work is
wholly independent of the other and can be processed without any
interactions. Generally, the goals of the work queue system are to ensure
that each piece of work is processed within a certain amount of time.
Workers are scaled up or scaled down to ensure that the work can
be handled. An illustration of a generic work queue is shown in Figure 10-1.
Figure 10-1. A generic work queue
A Generic Work Queue System
The work queue is an ideal way to demonstrate the power of distributed
system patterns. Most of the logic in the work queue is wholly independent
of the actual work being done, and in many cases the delivery of the work
can be performed in an independent manner as well. To illustrate this
point, consider the work queue illustrated in Figure 10-1. If we take
a look at it again and identify functionality that can be provided by
a shared set of library containers, it becomes apparent that most of
the implementation of a containerized work queue can be shared across
a wide variety of users, as shown in Figure 10-2.
Figure 10-2. The same work queue as shown in Figure 10-1, but this time using reusable containers. ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month, and much more.
