README.md 3.06 KB
Newer Older
Xavier Thompson's avatar
Xavier Thompson committed
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47
# A Comparison program between Cython+, Python and Rust

As a way to write a Cython+ program under real conditions, we ported an existing small program already written in Python and Rust to Cython+.

The program was chosen because it is short and because it is very conducive to parallel execution, allowing us to try out Cython+'s approach to concurrent programming.

## What it does

The program in question scans a filesystem from the root down (without following symbolic links) and:
- stores the results of a `stat` system call for each directory, file and symbolic link,
- computes messages digests from each file using `md5`, `sha1`, `sh256` and `sha512` algorithms,
- stores the target path of each symbolic link,

A JSON representation of the gathered information is then dumped in a file.

As Cython+ lacks an argument parser, the root directory is hardcoded as `/` and the output file as `result.json`.

The Python and Rust programs originally were making use of an argument parser, gathering additional information, and then uploading the result online, but they were simplified to match the description above because Cython+ currently lacks a good standard library, substantially increasing the required development work for even seemingly basic tasks.


## Building the Cython+ version

### Dependencies:

The program works on linux with Python3.7 and depends on the following:

- openssl library
- [fmt library](https://fmt.dev/latest/index.html)
- python3.7 development headers, which depending on the distribution may be packaged separately from python itself
- [Nexedi's cython+ compiler](https://lab.nexedi.com/nexedi/cython)

The paths to the openssl and fmt libraries should be updated in `cython/Makefile` and `cython/setup.py`.
The path to the python development headers shoudl be updated in `cython/Makefile`.
The path to the cython+ compiler should be added to environment variable `PYTHONPATH`.


### Building

First go to the cython subdirectory:

```
$ cd cython/
```

To build:

```
48
$ make
Xavier Thompson's avatar
Xavier Thompson committed
49 50
```

51
Or equivalently: `python3 setup.py build_ext --inplace`
Xavier Thompson's avatar
Xavier Thompson committed
52 53 54 55

To run:

```
56
$ make run
Xavier Thompson's avatar
Xavier Thompson committed
57 58 59 60 61
```


### Building without the Python runtime

62
The Cython compiler systematically links compiled programs against the python runtime, however this program never actually calls into the Python runtime.
Xavier Thompson's avatar
Xavier Thompson committed
63

64
There is no support yet in the Cython+ language to express independance from the Python runtime and generate code which does not need to include the Python headers. However in the meantime we introduce a hack: we only use the cython compiler to generate a C++ file which we then compile and link manually, but instead of providing the python runtime to link against we tell the linker to ignore linking errors. We also provide a `main` function as an entry point to bypass the entry point for the python interpreter.
Xavier Thompson's avatar
Xavier Thompson committed
65

66 67 68
This hack can only work because this program happens not to require the python runtime for anything.
No guarantees whatsoever are made that the program produced will behave as expected.
It has been tested and shown to work with g++.
Xavier Thompson's avatar
Xavier Thompson committed
69

70
To build:
Xavier Thompson's avatar
Xavier Thompson committed
71 72

```
73
$ make nopython
Xavier Thompson's avatar
Xavier Thompson committed
74 75 76 77 78
```

To run:

```
79
$ make runnopython
Xavier Thompson's avatar
Xavier Thompson committed
80
```