Patterns of Distributed Systems

Chapter 32

Request Pipeline

Improve latency by sending multiple requests on the connection without waiting for the response of the previous requests.

Problem

Communicating between servers within a cluster using Single-Socket Channel can cause performance issues if requests need to wait for responses to previous requests. To achieve better throughput and latency, the request queue on the server should be filled enough to make sure the server capacity is fully utilized. For example, when Singular Update Queue is used within a server, it can always accept more requests, until the queue fills up, while it’s processing a request. If only one request is sent at a time, most of the server capacity is unnecessarily wasted.

Solution

Nodes send requests ...