One of our clients had an overnight batch problem. They would kick off a big job every midnight to summarize the prior day’s data and prepare the system for business at 8:00 a.m. But the job had been taking longer and longer, and now it was running until 8:30, through the end of its window. They were paying fines to their own customers, for not having their system ready by 8:00 sharp.
To understand how this complex collection of overnight processes was spending our client’s time, we used a type of bar chart called a Gantt chart. Gantt charts help you see where you’re wasting time. They are another tool for helping you look at the right it. On a Gantt chart, time flows from left to right. The bars of a Gantt chart represent work being done, and the connector lines represent dependencies.
In our Gantt chart, we tracked where the database had spent its time overnight. It contained several occurrences of a pattern that looked like this:
Seeing the process flow illustrated this way led several of us to ask, all at the same time, “What’s happening in the time gap between loading and then processing that load? Why is the database not busy there?!” There was plenty of work to be done; why was the database not doing it? It’s the question that cracked the case.
It turned out to be a performance improvement opportunity in the application, not the database. The ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month, and much more.
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.
Julian F.
Head of Cybersecurity
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.
Addison B.
Field Engineer
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.
Amir M.
Data Platform Tech Lead
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.
