I/O batching and throttling
It is well known that chatty I/O calls generally lead to poor performance. In general, the solution is to batch together several messages and send them in one payload. In databases and network calls, batching is a common and useful technique to improve throughput. On the other hand, large batch sizes may actually harm throughput as they tend to incur memory overhead, and components may not be ready to handle a large batch at once. Hence, sizing the batches and throttling are just as important as batching. I would strongly advise conducting your own tests to determine the optimum batch size under representative load.
JDBC batch operations
JDBC has long had batch-update support in its API, which includes the INSERT
, UPDATE ...
Get Clojure High Performance Programming - Second Edition now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.