Take a Step Back
Looking at the profile again, we think we collected all the low-hanging fruit there. Row parsing is still slow, though. The first five rows in the flat profile are for functions that do that parsing. In short, we optimized individual functions, but the whole thing is still slow.
In such cases the next step in optimization is to look at the code at a higher abstraction level. In our case we’ll need to reexamine what the parsing code does. The first thing we note is that our code doesn’t take advantage of the fact that column order is predefined. The first column is always a number, the second a name, and the third a date. So instead of determining the column type in the if/else statement, we can just go ahead and parse it. ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access