So far we’ve examined systems that replicate for scalability in terms of
the number of requests processed per second (the stateless replicated
pattern), as well as scalability for the size of the data (the sharded data
pattern). In this chapter we introduce the scatter/gather pattern, which uses replication for scalability in terms of time. Specifically, the
scatter/gather pattern allows you to achieve parallelism in servicing
requests, enabling you to service them significantly faster
than you could if you had to service them sequentially.
Like replicated and sharded systems, the scatter/gather pattern is a tree
pattern with a root that distributes requests and leaves that process
those requests. However, in contrast to replicated and sharded systems, scatter/gather requests are simultaneously farmed out to all of the
replicas in the system. Each replica does a small amount of processing
and then returns a fraction of the result to the root. The root server
then combines the various partial results together to form a single
complete response to the request and then sends this request back out
to the client. The scatter/gather pattern is illustrated in Figure 8-1.
Figure 8-1. A scatter/gather pattern
Scatter/gather is quite useful when you have a large amount of mostly independent processing that is needed to handle a particular request. With such problems, ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month, and much more.
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.
Julian F.
Head of Cybersecurity
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.
Addison B.
Field Engineer
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.
Amir M.
Data Platform Tech Lead
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.
