Cython

Chapter 9. Cython Profiling Tools

I’ve never been a good estimator of how long things are going to take.

— D. Knuth

If you optimize everything, you will always be unhappy.

— D. Knuth

Cython lets us easily move across the boundary between Python and C. Rather than taking this as license to bring in C code wherever we like, however, we should consider just how much C we want to mix with our Python. When we are wrapping a library, the answer is usually determined for us: we need enough C to wrap our library, and enough Python to make it nice to use. When we’re using Cython to speed up a Python module, though, the answer is much less clear. Our goal is to bring in enough C code to get the best results for our efforts, and no more. Cython has tools that can help us find this sweet spot, which we cover in this chapter.

Cython Runtime Profiling

When we are optimizing Cython code, the principles, guidelines, and examples in the rest of this book help us answer the how. But sometimes the challenge is determining what code needs to change in the first place. I strongly recommend that, rather than looking at code and guessing, we use profiling tools to provide data to quantify our programs’ performance.

Python users are spoiled when it comes to profiling tools. The built-in profile module (and its faster C implementation, cProfile) makes runtime profiling easy. On top of that, the IPython interpreter makes profiling nearly effortless with the %timeit and %run magic commands, which support profiling ...