How to scale this application
The example we created in this chapter uses two parts that work separately and use RabbitMQ to exchange tasks and results.
Scaling this application is very easy—you just need to start as many workers as you want, and they don't have to work on the same server. You can run workers on a distributed cluster of servers. If your workers can't handle a load, you can start extra workers that will consume waiting tasks immediately and start the decoding process.
But this system has a potential bottleneck – the message broker. For this example, you can simply handle it if you start extra independent message brokers; for example, RabbitMQ support multiple instances with its clustering feature. Your server can have multiple ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access