README.rst 5.24 KB
Newer Older
1 2
Cython+ - Multi-core concurrent programming in Python
======================================================
gabrieldemarmiesse's avatar
gabrieldemarmiesse committed
3

Stefane Fermigier's avatar
Stefane Fermigier committed
4
.. warning::
5

Stefane Fermigier's avatar
Stefane Fermigier committed
6 7
    This is the README for the `Cython+ <https://cython.plus/>`_.
    Since Cython+ is heavily based on `Cython <https://cython.org/>`_,
8
    you may want to read also the `README for Cython <./README-Cython.rst>`_.*
Stefane Fermigier's avatar
Stefane Fermigier committed
9 10


11 12
What's Cython+?
---------------
Stefane Fermigier's avatar
Stefane Fermigier committed
13

14 15 16 17 18 19 20 21 22 23 24 25
The aim of the "Cython+" project is to ensure that all the cores
of a microprocessor can be efficiently exploited with a program
written in Python in the field of system or network programming,
so as to correct the main shortcoming of the Python language and
increase the competitiveness of its ecosystem.  The envisaged
approach consists in transferring the extremely powerful multi-core
concurrent programming model of the Go language to the Python
language by relying on innovative scientific and technological
approaches stemming from three decades of French know-how in the
field of concurrent object programming around `Actalk
<http://www-poleia.lip6.fr/~briot/actalk/actalk.html>`_, and
leveraging the existing `Cython <https://cython.org/>`_ language.
gabrieldemarmiesse's avatar
gabrieldemarmiesse committed
26

27 28 29 30 31 32 33 34 35 36
Cython is actually a superset of the Python language that brings
together the strong typing of Python, performance equivalent to C
and a form of low-level parallelism well suited to scientific
computing. It is with Cython that the `scikit-learn
<https://scikit-learn.org/stable/>`_ libraries or certain components
of the NEO transactional distributed database are developed. Cython
corrects the shortcomings of the Python language in terms of typing
or performance. Cython also corrects the Global Interpreter Lock
(GIL) problem which is at the origin of the poor support of multi-core
microprocessors in Python.
gabrieldemarmiesse's avatar
gabrieldemarmiesse committed
37

38 39 40 41 42 43 44 45 46
In the "Cython+" project, we propose to remove the GIL in a very
specific way: only at the level of asynchronous Cython functions
not calling Python objects. So nothing is changed in the Python
language nor in the "CPython" runtime which is the reference
implementation of the Python language in C. All programs already
developed in Python remain compatible. We only modify the Cython
compiler and the subpart of the Cython language disjoint from Python,
which we extend with a garbage collector and coroutines that can
be used on a multi-core architecture.
gabrieldemarmiesse's avatar
gabrieldemarmiesse committed
47

48 49 50 51 52 53 54 55 56
Thus, "Cython+" will offer the same kind of coroutine programming
as Go, the same level of parallelism, the same kind of memory
management, the same kind of performance, exception handling that
Go does not fully benefit from, a better concurrent programming
model than Go, a very well-stocked standard library with much broader
community support, and guaranteed memory isolation between threads.
"Cython+" will become an alternative to Go with many advantages,
strengthening the community of the leading development language
that Python has become.
gabrieldemarmiesse's avatar
gabrieldemarmiesse committed
57

58

Stefane Fermigier's avatar
Stefane Fermigier committed
59 60
Installation and basic use
--------------------------
gabrieldemarmiesse's avatar
gabrieldemarmiesse committed
61

Stefane Fermigier's avatar
Stefane Fermigier committed
62
Simply run::
gabrieldemarmiesse's avatar
gabrieldemarmiesse committed
63

Stefane Fermigier's avatar
Stefane Fermigier committed
64
    pip install cython-plus
gabrieldemarmiesse's avatar
gabrieldemarmiesse committed
65

Stefane Fermigier's avatar
Stefane Fermigier committed
66 67
Then you can use the habitual Cython commands: ``cython``, ``cythonize`` and
``cygdb``, as well as import the ``cython`` module from your Python code.
gabrieldemarmiesse's avatar
gabrieldemarmiesse committed
68

Stefane Fermigier's avatar
Stefane Fermigier committed
69 70 71 72
It should work on Linux with Python 3.7 to 3.10.

**It doesn't (currently) work on MacOS or Windows.**

Stefane Fermigier's avatar
Stefane Fermigier committed
73 74 75

Documentation
-------------
gabrieldemarmiesse's avatar
gabrieldemarmiesse committed
76

77
- Project Website: <https://www.cython.plus/>
Stefane Fermigier's avatar
Stefane Fermigier committed
78 79 80 81 82 83 84 85

- Documentation:

  - `Motivation <https://www.cython.plus/P-CYP-Documentation.Motivation>`_
  - `Basic Syntax (by example) <https://www.cython.plus/P-CYP-Documentation.Basic.Syntax>`_
  - `Interacting with Python (by example) <https://www.cython.plus/P-CYP-Documentation.Interacting.With.Python>`_
  - `Concurrency <https://www.cython.plus/P-CYP-Documentation.Concurrency>`_

86
- Blog posts and articles:
Stefane Fermigier's avatar
Stefane Fermigier committed
87 88 89 90

  - `Automatic multithreaded-safe memory managed classes in Cython <https://www.nexedi.com/blog/NXD-Document.Blog.Cypclass>`_
  - `HowTo Use Cython+ in Jupyter Notebook <https://www.cython.plus/P-CYP-Howto.Jupyter>`_

91
- Sandbox (various code snippets and benchmark to help you get started): <https://github.com/abilian/cythonplus-sandbox>
92

93

Stefane Fermigier's avatar
Stefane Fermigier committed
94 95
Development
-----------
96

97 98
- Project repository: <https://lab.nexedi.com/nexedi/cython>
- Alternate repository (read-only): <https://github.com/abilian/cythonplus>
Stefane Fermigier's avatar
Stefane Fermigier committed
99
- CI: <https://github.com/abilian/cythonplus/actions>
gabrieldemarmiesse's avatar
gabrieldemarmiesse committed
100 101


Stefane Fermigier's avatar
Stefane Fermigier committed
102 103
License & Copyright
-------------------
gabrieldemarmiesse's avatar
gabrieldemarmiesse committed
104

105
Cython+ is a (friendly) fork of `Cython <https://cython.org/>`_.
gabrieldemarmiesse's avatar
gabrieldemarmiesse committed
106

107
Its copyright belongs to the Cython original authors (as listed
108 109
`Here <https://cython.org/#community>`_) as well as to the members of the
`Cython+ consortium <https://www.cython.plus/consortium/>`_: `Nexedi
110 111 112
<https://nexedi.com/>`_, `Abilian <https://abilian.com/>`_, `Teralab
<https://www.teralab-datascience.fr/?lang=en>`_ and `Inria
<https://inria.fr/>`_.
gabrieldemarmiesse's avatar
gabrieldemarmiesse committed
113

114
Cython+ is licensed under the permissive **Apache License**. See `LICENSE.txt <./LICENSE.txt>`_.
gabrieldemarmiesse's avatar
gabrieldemarmiesse committed
115 116


Stefane Fermigier's avatar
Stefane Fermigier committed
117 118
Contributing
------------
gabrieldemarmiesse's avatar
gabrieldemarmiesse committed
119

120
Want to contribute to the Cython+ project?
gabrieldemarmiesse's avatar
gabrieldemarmiesse committed
121

122
Please contact us at <https://www.cython.plus/contact/>.
Stefane Fermigier's avatar
Stefane Fermigier committed
123 124 125 126 127 128 129 130


Acknowledgements
----------------

Cython+ is based on the work of the `Cython authors <https://cython.org/#community>`_.

The Cython+ project has been selected to receive funding from the PSPC-Régions 1.
131 132
It is supported by `Cap Digital <https://capdigital.com/>`_ and financed by
the `PIA <https://www.gouvernement.fr/le-programme-d-investissements-d-avenir>`_ and the `Paris Region <https://www.iledefrance.fr/>`_.