Request Batch

Combine multiple requests to optimally utilise the network

Problem

If a lot of requests are sent to cluster nodes with a small amount of data, network latency and the request processing time (including serialization and deserialization of the request on the server side) can add significant overhead.

For example, in a network with 1gbps capacity, if the latency and request processing time is 100 microseconds and the client is sending hundreds of requests at the same time, it will significantly limit the overall throughput even though each request is just a few bytes.

Solution

Combine multiple requests together into a single request batch. The batch of the request will be sent to the cluster node for processing, with each request processed in exactly the same manner as an individual request. The node will then respond with a batch of responses.

for more details go to Chapter 31 of the online ebook at oreilly.com

This pattern is part of Patterns of Distributed Systems

23 November 2023