18
Queuing Tasks with Redis
Receiving requests, performing an action, and then returning a response to the user can solve a lot of problems in web programming. However, there are times when this simple approach will simply not cut it. For instance, when I was working at MonolithAi, we had a functionality where the user would be able to put in data and parameters and then train a machine learning model on that data at a click of a button. However, trying to train a machine learning model before sending a response to the user would simply take too long. The connection would probably time out. To solve this, we had a Redis queue and a pool of workers consuming tasks. The training task would be put into the queue and one of the workers would work ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access