High Performance Python

Chapter 7. Compiling to C

Questions You’ll Be Able to Answer After This Chapter

• How can I have my Python code run as lower-level code?
• What is the difference between a JIT compiler and an AOT compiler?
• What tasks can compiled Python code perform faster than native Python?
• Why do type annotations speed up compiled Python code?
• How can I write modules for Python using C or Fortran?
• How can I use libraries from C or Fortran in Python?

The easiest way to get your code to run faster is to make it do less work. Assuming you’ve already chosen good algorithms and you’ve reduced the amount of data you’re processing, the easiest way to execute fewer instructions is to compile your code down to machine code.

Python offers a number of options for this, including pure C-based compiling approaches like Cython, Shed Skin, and Pythran; LLVM-based compiling via Numba; and the replacement virtual machine PyPy, which includes a built-in just-in-time (JIT) compiler. You need to balance the requirements of code adaptability and team velocity when deciding which route to take.

Each of these tools adds a new dependency to your toolchain, and additionally Cython requires you to write in a new language type (a hybrid of Python and C), which means you need a new skill. Cython’s new language may hurt your team’s velocity, as team members without knowledge of C may have trouble supporting this code; in practice, though, this is probably a small concern as you’ll only use Cython in well-chosen, small regions ...