Python Cookbook

Tracking Instances of Particular Classes

Credit: David Ascher, Mark Hammond

Problem

You’re trying to track down memory usage of specific classes in a large system, and Recipe 14.10 either gives too much data to be useful or fails to recognize cycles.

Solution

You can design the constructors of suspect classes to keep a list of weak references to the instances in a global cache:

tracked_classes = {}
import weakref
def logInstanceCreation(instance):
    name = instance._ _class_ _._ _name_ _
    if not tracked_classes.has_key(name):
        tracked_classes[name] = []
    tracked_classes[name].append(weakref.ref(instance))

def reportLoggedInstances(classes): # "*" means all known instances
    if classes == '*':
        classes = tracked_classes.keys(  )
    else:
        classes = classes.split(  )
    classes.sort(  )
    for classname in classes:
        for ref in tracked_classes[classname]:
            ob = ref(  )
            if ob is not None:
                print ref(  )

To use this code, add a call to logInstanceCreation(self) to the _ _init_ _ calls of the classes whose instances you want to track. When you want to find out which instances are currently alive, call reportLoggedInstances( ) with the name of the classes in question (e.g., MyClass._ _name_ _).

Discussion

Tracking memory problems is a key skill for developers of large systems. The above code was dreamed up to deal with memory allocations in a system that involved three different garbage collectors; Python was only one of them. Due to the references between Python objects and non-Python objects, none of the individual ...