Python Cookbook

Debugging the Garbage-Collection Process

Credit: Dirk Holtwick

Problem

You know that memory is leaking from your program, but you have no indication of what exactly is being leaked. You need more information to help you figure out where the leaks are coming from, so you can remove them and lighten the garbage-collection work periodically performed by the standard gc module.

Solution

The gc module lets you dig into garbage-collection issues:

import gc

def dump_garbage(  ):
    """
    show us what the garbage is about
    """
    # Force collection
    print "\nGARBAGE:"
    gc.collect(  )

    print "\nGARBAGE OBJECTS:"
    for x in gc.garbage:
        s = str(x)
        if len(s) > 80: s = s[:77]+'...'
        print type(x),"\n  ", s

if _ _name_ _=="_ _main_ _":
    gc.enable(  )
    gc.set_debug(gc.DEBUG_LEAK)

    # Make a leak
    l = []
    l.append(l)
    del l

    # show the dirt ;-)
    dump_garbage(  )

Discussion

In addition to the normal debugging output of gc, this recipe shows the garbage objects to help you get an idea of where the leak may be. Situations that could lead to garbage collection should be avoided. Most of the time, they’re caused by objects that refer to themselves, or similar reference loops (also known as cycles).

Once you’ve found where the reference loops are coming from, Python offers all the needed tools to remove them, particularly weak references (in the weakref standard library module). But especially in big programs, you first have to get an idea of where to find the leak before you can remove it and enhance your program’s performance. For ...