Chapter 6. Rate Limiting
Much of what I’ve covered so far has to do with the things you can do on the client side to deal with problems getting an appropriate response from a server. Often when a client is struggling to get a response, it’s because the server it’s talking to has too much work to do and is overwhelmed.
In this chapter, I’ll be looking at approaches to reducing the amount of work being processed, a technique called rate limiting. As you’ll see, rate limiting can be implemented in numerous ways, and I’ll be sharing some concrete patterns that can help you implement rate limiting in your own system.
Before I get into the details, though, let’s step back a bit and consider what your options are when a server has too much work.
Ways to Handle Having Too Much Work
When a server has so much work to do that it is in danger of falling over as a result, you have five options to consider:
-
Just let it fall over.
-
Throw away some of the work (load shedding).
-
Reduce the amount of work being sent (backpressure). ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access