Powerful Command-Line Applications in Go

Reducing Memory Allocation

You now know that the tool spends a lot of time with the garbage collector because it’s allocating too much memory. You also know that the bulk of the memory allocation comes from the ReadAll function of the encoding/csv package, which you’re calling from the csv2float function.

If you follow the program’s logic, it’s reading all the records from each CSV file into memory and then processing them one at a time with a loop, storing the result in another slice. It’s not doing anything that requires the entire content of the file to be in memory, so you can replace the calls to ReadAll with the function Read from the encoding/csv package. This function reads one record at a time, so you can execute it directly inside ...