It’s important to know that queueing bends the response time curve into a hockey stick. Mathematically, it’s a hyperbola. This shape is what defines the nature of queueing: response times will vary gently on systems with low traffic intensities, and they’ll vary wildly on systems with high traffic intensities. Of course, big response time variances make interactive users especially miserable.
Your service channel count determines the bend in your hyperbola. M/M/1 systems have noticeable response time decay even at low traffic intensities. M/M/128 systems have virtually no decay throughout a broad range of traffic intensities, but then they decay violently at loads near ρ = 1.
Expensive systems like M/M/128, whose R values stay flat longer as you add load, are what you’ll use to handle ginormous high-concurrency workloads. But even the most scalable system on the planet will make you queue forever if you push its traffic intensity hard enough. That’s why you need to stay a safe distance away from that skyward-pointing, righthand side of the curve, no matter what kind of system you’re using.
What is a “safe distance”? You’ll want to operate your system in the range of loads where tiny fluctuations in traffic intensity cause only tiny variances in response times. If there’s so much load on your system that you can’t do that, then you’ll need to either eliminate ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month, and much more.
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.
Julian F.
Head of Cybersecurity
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.
Addison B.
Field Engineer
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.
Amir M.
Data Platform Tech Lead
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.
