Python Cookbook

Module: Allowing the Python Profiler to Profile C Modules

Credit: Richie Hindle

Profiling is the most crucial part of optimization. What you can’t measure, you cannot control. This definitely applies to your program’s running time. To make sure that Python’s standard profile module can also measure the time spent in C-coded extensions, you need to wrap those extensions with the module shown in Example 15-1. An alternative to the approach in this module is to use the new Hotshot profiler that ships with Python 2.2 and later.

This module lets you take into account time spent in C modules when profiling your Python code. Normally, the profiler profiles only Python code, so it’s difficult to find out how much time is spent accessing a database, running encryption code, sleeping, and so on. This module makes it easy to profile C code as well as Python code, giving you a clearer picture of how your application is spending its time.

This module also demonstrates how to create proxy objects at runtime that intercept calls between preexisting pieces of code. Furthermore, it shows how to use the new module to create new functions on the fly. We could do many of these things in a somewhat lightweight fashion, but systematically using the new module is a good way to demystify its reputation for difficulty.

Here’s a small piece of code using the rotor encryption module:

import rotor, profile
r = rotor.newrotor('key')
profile.run("r.encrypt('Plaintext')")

This won’t produce any profiler output ...