Python Cookbook

Implementing a Simple Extension Type

Credit: Alex Martelli

Problem

You want to code and build a C extension type for Python with a minimal amount of hard work.

Solution

First of all, we need to create a setup.py using the distutils package (in Python 2.0 and later) to build and install our module:

from distutils.core import setup, Extension

setup(name = "elemlist",
      version = "1.0",
      maintainer = "Alex Martelli",
      maintainer_email = "[e-mail]",
      description = "Sample, simple Python extension module",

      ext_modules = [Extension('elemlist',sources=['elemlist.c'])]
)

Then we need an elemlist.c file with our module’s source code:

#include "Python.h"

/* type-definition and utility-macros */
typedef struct {
    PyObject_HEAD
    PyObject *car, *cdr;
} cons_cell;
staticforward PyTypeObject cons_type;
/* a type-testing macro (we don't actually use it here) */
#define is_cons(v) ((v)->ob_type == &cons_type)
/* utility macros to access car and cdr, both as lvalues and rvalues */
#define carof(v) (((cons_cell*)(v))->car)
#define cdrof(v) (((cons_cell*)(v))->cdr)

/* ctor ("internal" factory function) and dtor */
static cons_cell*
cons_new(PyObject *car, PyObject *cdr)
{
    cons_cell *cons = PyObject_NEW(cons_cell, &cons_type);
    if(cons) {
        cons->car = car; Py_INCREF(car); /* INCREF when holding a PyObject */
        cons->cdr = cdr; Py_INCREF(cdr); /* ditto */
    }
    return cons;
}
static void
cons_dealloc(cons_cell* cons)
{
    /* DECREF when releasing previously held PyObject*'s */
    Py_DECREF(carof(cons)); Py_DECREF(cdrof(cons));
    PyObject_DEL(cons);
}

/* The Python type-object */
statichere PyTypeObject cons_type = {
    PyObject_HEAD_INIT(0)    /* initialize to 0 to ensure Win32 portability */
    0,                 /*ob_size*/
    "cons",            /*tp_name*/
    sizeof(cons_cell), /*tp_basicsize*/
    0,                 /*tp_itemsize*/
    /* methods */
    (destructor)cons_dealloc, /*tp_dealloc*/
    /* implied by ISO C: all zeros thereafter, i.e., no other method */
};

/* module functions */
static PyObject*
cons(PyObject *self, PyObject *args)    /* the exposed factory function */
{
    PyObject *car, *cdr;
    if(!PyArg_ParseTuple(args, "OO", &car, &cdr))
        return 0;
    return (PyObject*)cons_new(car, cdr);
}
static PyObject*
car(PyObject *self, PyObject *args)     /* car accessor */
{
    PyObject *cons;
    if(!PyArg_ParseTuple(args, "O!", &cons_type, &cons)) /* type-checked */
        return 0;
    return Py_BuildValue("O", carof(cons));
}
static PyObject*
cdr(PyObject *self, PyObject *args)     /* cdr accessor */
{
    PyObject *cons;
    if(!PyArg_ParseTuple(args, "O!", &cons_type, &cons)) /* type-checked */
        return 0;
    return Py_BuildValue("O", cdrof(cons));
}
static PyMethodDef elemlist_module_functions[] = {
    {"cons",   cons,   METH_VARARGS},
    {"car",    car,    METH_VARARGS},
    {"cdr",    cdr,    METH_VARARGS},
    {0, 0}
};

/* module entry point (module initialization) function */
void
initelemlist(void)
{
    /* Create the module with its functions */
    PyObject *m = Py_InitModule("elemlist", elemlist_module_functions);
    /* Finish initializing the type objects */
    cons_type.ob_type = &PyType_Type;
}

Discussion

C-coded Python extension types have an undeserved aura of mystery and difficulty. Sure, it’s a lot of work to implement every possible nicety, but a fundamental, useful type doesn’t take all that much effort.

This module is roughly equivalent to the Python-coded module:

def cons(car, cdr): return car, cdr
def car(conscell): return conscell[0]
def cdr(conscell): return conscell[1]

except that the C version contains about 25 times more lines of code, even excluding comments and empty lines (and it is not much faster than the Python-coded version, either).

However, the point of this recipe is to demonstrate a minimal C-coded extension type. I’m not even supplying object methods (except the necessary destructor) but, rather, module-level functions for car and cdr access. This also shows the utter simplicity of building a C-coded extension module on any platform, thanks to the distutils package, which does all of the hard work.

Because this is meant as an introduction to writing extension modules in C for Python, here are the instructions on how to build this extension module, assuming you have a Windows machine with Python 2.0 or later, and Microsoft Visual C++ 6 (or the free command-line equivalent that you can download from Microsoft’s site as a part of their .NET Framework SDK). You can presumably translate mentally to other platforms such as Linux with gcc, for example. On the other hand, using non-Microsoft compilers on Windows takes more work, and I’m not going to cover that here (see http://www.python.org/doc/current/inst/non-ms-compilers.html). The steps are:

1. Make a new directory, C:\Temp\EL, for example.

2. Open a command prompt (MS-DOS box) and go to the new directory.

3. In the new directory, create the files setup.py and elemlist.c with the contents of the recipe’s text.

4. Run the following at the DOS prompt (assuming you’ve done a standard Python install, C:\Python22 is where your python.exe lives):

   C:\Temp\EL> C:\Python22\python setup.py install

   This will give lots of output, but presumably, all goes well and the new extension has been built and installed.

5. Now test it by running the following at the DOS prompt:

   C:\Temp\EL> C:\Python22\python
   snipped—various greeting messages from Python
   >>> from elemlist import cons
   >>> a=cons(1,cons(2,cons(3,(  ))))
   >>> from elemlist import car, cdr
   >>> car(cdr(a))
   2

Now your new extension module is installed and ready!

See Also

The Extending and Embedding manual is available as part of the standard Python documentation set at http://www.python.org/doc/current/ext/ext.html; the Distributing Python Modules section of the standard Python documentation set is still incomplete, but it is the best source of information on the distutils package.