Ctrl-C

Interfacing C and Python 3 using SWIG

I like Python, and I use it for almost all of my personal projects and whenever I can get away with it in school. Sometimes, however, (rarely, these days) you’re looking for raw speed. Python, alas, is not the fastest kid on the block, and at these times you might have to turn to a different language to give you the necessary speed. (Another option you might consider first is Psyco.) In these situations it is often the case that only particular operations or functions really have to be fast. Even if you want to write the whole piece of software in C, you might like to use Python to write the front end. There are a lot of different ways to do this, but my favorite is SWIG, because it’s easy and I’m lazy.

SWIG is a great tool that supports a host of high-level languages; I’ve only ever used it with Python. At the other end, you get your choice of C and C++—crappy languages aren’t really my thing, so this tutorial will be about C. I’m going to use Python 3 because all the cool kids live on the bleeding edge. This requires SWIG 1.3.37 or better; if you’re on Ubuntu like me, you’re going to have to go grab it from the website and compile it yourself (I’m using 1.3.39). Also, remember that you need the header files for python installed (python3-dev on Ubuntu). If you’re too lazy to copy-paste, you can download the files here.

We’ll start out with the obligatory hello world program just to test that SWIG is set up correctly. First make a C function that prints the text and name it hello.c:

#include<stdio.h>
 
void hello() {
    printf("Hello, World!\n"):
}

Now we’ll get SWIG to generate a wrapper for us so that we can call it in Python. Make a file called hello.i and put this in it (my code highlighting isn’t feeling very confident about SWIG files, sorry):

%module hello
%{
    void hello();
%}

This file is the interface file, and it tells SWIG what functions and data structures are going to be shared between C and Python. Now we’ll have SWIG process the interface file to make a wrapper for a python module:

$ swig -python hello.i

This should generate two files: hello.py and hello_wrap.c. Compile the C files:

$ gcc -c -fPIC hello_wrap.c hello.c -I/usr/include/python3.0 -I/usr/lib/python3.0

Make sure that the direcotories are correct. Unfortunately, gcc generates a warning when I compile, which is bothersome as I’m a -Werror kind of guy. The -fPIC flag is necessary, because we’re making a shared library the python module is going to use:

$ ld -shared hello.o hello_wrap.o -o _hello.so

If that works, you are ready to fire up Python and try it. At a Python3 prompt:

>>> from hello import *
>>> hello()
Hello, World!

Easy, right? Now I’ll show you some of the features that I most frequently use. First, we’ll pass some data through by wrapping a C function to multiply integers. Make a file called swig_intro.h, and put in the following declaration:

#include<stdlib.h>
#include<stdio.h>
 
int mult(int a, int b);

Then in swig_intro.c we can write

#include "swig_intro.h"
 
int mult(int a, int b) {
    return a * b;
}

Finally, the interface file, swig_intro.i, will look like this:

%module swig_intro
%{
    #include "swig_intro.h"
%}
 
int mult(int a, int b);

To make things easier, let’s write a makefile for this little project:

TARGET     = swig_intro
CC         = gcc
CFLAGS     = -g -fPIC -Wall -Wextra
HEADERS    = swig_intro.h
SRCS       = swig_intro.c
INCLUDES   =
INTERFACE  = swig_intro.i
SWIG       = swig
 
LDSHARED   = ld
CSHARED    = -shared
 
OBJS       = $(SRCS:.c=.o)
 
IWRAP      = $(INTERFACE:.i=_wrap.i)
ISRCS      = $(IWRAP:.i=.c)
IOBJS      = $(IWRAP:.i=.o)
 
PYTHON_INCLUDE= -I/usr/include/python3.0 -I/usr/lib/python3.0
PYTHON_SO     = .so
 
all: $(SRCS)
	$(SWIG) -python $(INTERFACE)
	$(CC) -c $(CCSHARED) $(CFLAGS) $(ISRCS) $(HEADERS) $(SRCS) $(INCLUDES) $(PYTHON_INCLUDE)
	$(LDSHARED) $(CSHARED) $(OBJS) $(IOBJS) -o _$(TARGET)$(PYTHON_SO)
 
clean:
	rm -f *_wrap* *~ .~* *.pyc
	rm -f *.o *.so *.so

If that works, you can launch a Python3 shell (from now on I’ll assume you do the import first):

>>> from swig_intro import *
>>> mult(-10, 5)
-50

Behind the scenes, SWIG uses something called typemaps to specify how each type is from Python to C and vice versa. In many cases, such as this, we don’t need to worry about that at all, because SWIG has default typemaps for all the fundamental datatypes. For example, you might wonder what SWIG does with a C struct. Go ahead and add this definition to the header file and the interface file:

typedef struct Person {
    char *first_name;
    char *last_name;
    int birth_year;
} Person;
 
void print_person(Person *p);

and put the following function into the .c file:

void print_person(Person *p) {
    if (!p->first_name || !p->last_name) {
        printf("The person is not properly defined.\n");
        return;
    }
    printf("%s %s was born in %d.\n", p->first_name, p->last_name, p->birth_year);
}

Now recompile and try this:

>>> p = Person()
>>> p.first_name = "Paul"
>>> p.last_name = "Black"
>>> p.birth_year = 1977
>>> print_person(p)
Paul Black was born in 1977.

As you can see, the the struct Person was wrapped in a very simple Python class with the three public data fields first_name, last_name, and birth_year.

Next, let’s see how to use typemaps ourselves to pass a more complex structure from Python to C. We’re going to make a (rather contrived) C function that will return the product of a list of integers. We’ll do this by turning a Python list into a C struct that specifies an int array and a size. Add the following code to the header and interface files:

struct IntList {
    int size;
    long *list;
} IntList;
 
int product(IntList *l);

In the .c file, add the function:

int product(IntList *l) {
    int i;
    if (l->size <= 0) {
        return 0;
    }
    int ret_val = 1;
    for (i = 0; i size; i++) {
        ret_val *= l->list[i];
    }
    return ret_val;
}

Now we need to define the typemap in the .i file. This isn’t too difficult; we just make use of several functions from the Python C API:

%typemap(in) IntList * {
    if (!PyList_Check($input)) {
        PyErr_SetString(PyExc_TypeError, "Not a list.");
        return NULL;
    }
    int size = PyList_Size($input);
    $1 = (IntList *)malloc(sizeof(IntList));
    long *list = (long *)malloc(sizeof(long) * size);
    $1->list = list;
    $1->size = size;
    int i;
    for (i = 0; i < size; i++) {
        PyObject *item = PyList_GetItem($input, i);
        if (!PyInt_Check(item)) {
            PyErr_SetString(PyExc_TypeError, "List item is not an integer.");
            free(list);
            free($1);
            return NULL;
        }
        list[i] = PyInt_AsLong(item);
    }
}
 
%typemap(freearg) IntList * {
    free(((IntList *)$1)->list);
    free((IntList *)$1);
}

This also illustrates how to tell SWIG to free your objects and how to handle errors. Now we can see it in action:

>>> from swig_intro import *
>>> product(5)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Not a list.
>>> product(["five"])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: List item is not an integer.
>>> product([1,3,5,7,9])
945

As you would expect, transferring data the other way is similar. For an example, let’s make a function that takes a list and returns a dictionary mapping the list elements to their squares. Add these definitions to the header and interface files:

struct IntMap {
    int size;
    long *list1;
    long *list2;
} IntMap;
 
IntMap *square(IntList *l);

The actual C function will take a pointer to an IntList and return a pointer to an IntMap, allocating space for a second list as needed. The conversion to a dict will be specified with an out typemap. In the .c file:

IntMap *square(IntList *l) {
    IntMap *ret_map = (IntMap *)malloc(sizeof(IntMap));
    ret_map->size = l->size;
    ret_map->list1 = l->list;
    ret_map->list2 = (long *)malloc(sizeof(long) * l->size);
    int i;
    for (i = 0; i size; i++) {
        ret_map->list2[i] = l->list[i] * l->list[i];
    }
    return ret_map;
}

Add the typemap to the .i file:

%typemap(out) IntMap * {
    $result = PyDict_New();
    int i;
    for (i = 0; i size; i++) {
        PyObject *key = PyInt_FromLong($1->list1[i]);
        PyObject *val = PyInt_FromLong($1->list2[i]);
        PyDict_SetItem($result, key, val);
    }
}
 
%typemap(freearg) IntMap * {
    free(((IntMap *)$1)->list2);
    free((IntMap *)$1);
}

And we’re done!

>>> square([1,-12,60000])
{60000: 3600000000, 1: 1, -12: 144}

I’ve only shown some of the more useful capabilities of SWIG that I find myself using a lot. For more information, consult the official SWIG docs. Unfortunately, they’re not entirely complete, so it is sometime easier to get answers by just browsing the built-in SWIG interface files to see how the typemaps work.

Tags: C, python, python 3, swig

1 comment to Interfacing C and Python 3 using SWIG