Cython backend aiming PyPy : Week 3

Last week, I worked on the implementation of pointers. It works for the global cases but special cases are not really handled : char* is converted into POINTER(c_char) instead of c_char_p (although I'm not sure if it's a big deal), however void* does not work.
I also extended Cython's python code writer to make it work with the backend. I only tested it for very basic example though.
Speaking of testing, I also looked at how Cython tests are written and I will now write automated tests, the code writer will help me do that as I think it's better to test what code the output code rather than the AST nodes.
For extern definitions unions are still to be implemented but I plan to do this later.
For this week, I plan to :

• Write some automated tests (I would need them anyway and it will be good to start keeping track of regressions)
• Experiment on calling C functions (in particular passing cdef'd variables as parameter of C functions)

EDIT: I also implemented the "linking" of functions

Cython backend aiming PyPy : Week 2

During this week, I worked on the transformation of the extern definitions :

• Functions definitions are turned into ctypes functions, functions can have basic C types and structures as parameters (but no pointers for the moment)
• Structures definitions are turned into ctypes structures, however, for the moment, structures need to be in the right order (a structure A used by structure B needs to be declared before B)
• Structures can be partially declared thanks to ctypes_configure. This dependency is only needed at compilation time and requires a C compiler (this assumes that the compiler running on the host machine compiles the program with a suitable alignment for the machine running the program).

On this part of the project I still have to :

• Implement pointers
• Implement forward declarations
• Implement unions

It will also be useful to implement a basic code writer as it will be easier to present my work and, it will improve testing as well.

URL of the repository

I uploaded my working code on github.
https://github.com/hardshooter/CythonCTypesBackend

For the moment :

• Addition of the --python command line switch
• Prints the C functions defined externally :
• "int func1(), func2()" is split into 2 definitions

Cython backend aiming PyPy : Week 1

During last week and the community bonding period, I tried to refine my subject by doing researches on what features of Cython can be handled and how I could implement those. To summarize, the backend aims to produce pure Python code (I decided to use ctypes to wrap C libraries as it will allow other Python implementations to use the backend). This introduces some restrictions :

• ctypes is not compatible with C++, thus the backend will not implement any C++ wrapping functionality
• We have to care about the ABI while the standard backend does only care about the API, ctypes can only call C functions so C macros and C global variables will have to be handled differently (probably by generating C functions returning the macros or, for some very explicit cases, inline the macro into the Python code)

On the implementation point of view, I'm currently writing a transformation which will turn extern function definition into variable assignment:

cdef extern from "foo.h":
    int bar(int, long)

will become:

bar = library.bar
bar.argtypes = [c_int, c_long]
bar.restype = c_int

This creates another problem, while the standard Cython backend leaves the linking phase to the C compiler, this backend will need the name of the C libraries to realise some kind of linking. For the following weeks, I plan to :

• finish the "cdef extern from" declaration transformation
• implement the "linking" phase
• turn cdef-ed variable into ctypes objects
• implement the pass by value mechanism

This will probably give me enough work for quite some time.