Expert Python Programming - Third Edition

Profiling memory usage

Another problem you may encounter when optimizing an application is memory consumption. If a program starts to eat so much memory that the system begins to continuously swap, there is probably a place in your application where too many objects are being created or objects that are not needed anymore are still kept alive by some unintended reference. This kind of resource mismanagement isn't easy to detect through typical CPU profiling techniques. Sometimes, consuming enough memory to make a system swap may involve a lot of CPU work that can be easily detected with ordinary profiling techniques. But usually, performance drop can happen suddenly and in an unexpected moment that is unrelated to the actual programming error. ...