Commit 2fcf3a39 authored by scoder's avatar scoder

Merge pull request #298 from delafont/master

Rewrote the 'Pure Python mode' tutorial 
parents 913d9a19 cfb9567b
...@@ -4,20 +4,139 @@ ...@@ -4,20 +4,139 @@
Pure Python Mode Pure Python Mode
================ ================
Cython provides language constructs to let the same file be either interpreted Sometimes one may want to speed up Python code without losing the possibility to run
or compiled. This is accomplished by the same "magic" module ``cython`` that it with the Python interpreter. While pure Python scripts can be compiled with Cython,
directives use and which must be imported. This is available for both :file:`.py` and it usually results in a 20%-50% speed gain only.
:file:`.pyx` files.
This is accomplished via special functions and decorators and an (optional) To go beyond that, Cython provides language constructs to add static typing
augmenting :file:`.pxd` file. and cythonic functionalities to a Python module to make it run much faster
when compiled, while still allowing it to be interpreted.
This is accomplished either via an augmenting :file:`.pxd` file, or
via special functions and decorators available after importing ``cython``.
Although it is not typically recommended over writing straight Cython code
to a :file:`.pyx` file, one can have specific reasons to do so -
easier testing, collaboration with pure Python developers, etc.
In pure mode, you are more or less restricted to code that can be expressed
(or at least emulated) in Python, plus static type declarations. Anything
beyond that can only be done in .pyx files with extended language syntax,
because it depends on compilation.
Augmenting .pxd
---------------
Using an augmenting :file:`.pxd` allows to let the original :file:`.py` file
completely untouched. On the other hand, one needs to maintain both
the :file:`.pxd` and the :file:`.py` in parallel.
Note that :file:`.pxd` files are used differently when they come together with
:file:`.py` than with :file:`.pyx` files (see :doc:`pxd_files`). Declarations
in a :file:`.pyx` must correspond to those of the :file:`.pxd`, whilst
declarations in a :file:`.py` file can be overridden/augmented by the more
specific ones present in a :file:`.pxd`.
If a :file:`.pxd` file is found with the same name as a :file:`.py` file,
it will be searched for :keyword:`cdef` classes and :keyword:`cdef`/:keyword:`cpdef`
functions and methods. It will then convert the corresponding
classes/functions/methods in the :file:`.py` file to be of the correct type.
Thus if one has a file :file:`A.py`::
def myfunction(x, y=2):
a = x-y
return a + x * y
class A:
def __init__(self, b=0):
self.a = 3
self.b = b
def foo(self, x):
print x + 1.0
and adds :file:`A.pxd`::
cpdef int myfunction(int x,int y)
cdef class A:
cdef public int a,b
cpdef foo(self, double x)
then at compilation time :file:`A.py` would be interpreted as::
cpdef int myfunction(int x,int y):
a = x-y
return a + x * y
cdef class A:
cdef public int a,b
def __init__(self, b=0):
self.a = 3
self.b = b
cpdef foo(self, double x):
print x + 1.0
while still letting the possibility of running the Python interpreter
as before with `python A.py`.
Notice how in order to provide the Python wrappers to the definitions
in the :file:`.pxd`, that is, to be accessible from Python,
* function signature declarations must be declared as `cpdef`::
cpdef int myfunction(int x,int y)
* function definitions must be declared as `cpdef inline`::
cpdef inline int myfunction(int x,int y):
pass
* `cdef` classes are declared as `cdef class`;
* `cdef` class attributes must be declared as `cdef public`;
* `cdef` class methods must be declared as `cpdef`.
Also in the example above, one cannot fix the type of the local variable `a`
used within `myfunction` with such definitions. For that purpose
one can use ``cython``'s ``@locals`` decorator (see :ref:`magic_attributes`, and
:ref:`magic_attributes_pxd`).
Normal Python (:keyword:`def`) functions cannot be declared in
:file:`.pxd` files, so it is currently impossible to override the types of
Python functions in :file:`.pxd` files if they use ``*args`` or ``**kwargs``
in their signature, for instance.
.. _magic_attributes:
Magic Attributes Magic Attributes
---------------- ----------------
The currently supported attributes of the ``cython`` module are: Special decorators are available using the ``cython`` module that can
be used to add static typing within the Python file, while being ignored
by the interpreter.
This option adds the ``cython`` dependency to the original code, but does
not require to maintain a supplementary file.
* ``declare`` declares a typed variable in the current scope, which can be used in "Compiled" switch
^^^^^^^^^^^^^^^^^
* ``compiled`` is a special variable which is set to ``True`` when the compiler
runs, and ``False`` in the interpreter. Thus the code::
if cython.compiled:
print("Yep, I'm compiled.")
else:
print("Just a lowly interpreted script.")
will behave differently depending on whether or not the code is loaded as a
compiled :file:`.so` file or a plain :file:`.py` file.
Static typing
^^^^^^^^^^^^^
* ``cython.declare`` declares a typed variable in the current scope, which can be used in
place of the :samp:`cdef type var [= value]` construct. This has two forms, the place of the :samp:`cdef type var [= value]` construct. This has two forms, the
first as an assignment (useful as it creates a declaration in first as an assignment (useful as it creates a declaration in
interpreted mode as well):: interpreted mode as well)::
...@@ -29,13 +148,27 @@ The currently supported attributes of the ``cython`` module are: ...@@ -29,13 +148,27 @@ The currently supported attributes of the ``cython`` module are:
cython.declare(x=cython.int, y=cython.double) # cdef int x; cdef double y cython.declare(x=cython.int, y=cython.double) # cdef int x; cdef double y
* ``locals`` is a decorator that is used to specify the types of local variables It can also be used to type class constructors::
class A:
cython.declare(a=cython.int, b=cython.int)
def __init__(self, b=0):
self.a = 3
self.b = b
* ``@cython.locals`` is a decorator that is used to specify the types of local variables
in the function body (including any or all of the argument types):: in the function body (including any or all of the argument types)::
@cython.locals(a=cython.double, b=cython.double, n=cython.p_double) @cython.locals(a=cython.double, b=cython.double, n=cython.p_double)
def foo(a, b, x, y): def foo(a, b, x, y):
n = a*b
... ...
It cannot be used to type class constructor attributes. See ``cython.declare``
instead to do so.
* ``@cython.returns(<type>)`` specifies the function's return type.
* Starting with Cython 0.21, Python signature annotations can be used to * Starting with Cython 0.21, Python signature annotations can be used to
declare argument types. Cython recognises three ways to do this, as declare argument types. Cython recognises three ways to do this, as
shown in the following example:: shown in the following example::
...@@ -46,6 +179,32 @@ The currently supported attributes of the ``cython`` module are: ...@@ -46,6 +179,32 @@ The currently supported attributes of the ``cython`` module are:
explicit_c_type: {'ctype': 'int'}): explicit_c_type: {'ctype': 'int'}):
... ...
Extension types and cdef functions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* ``@cython.cclass`` creates a ``cdef class``.
* ``@cython.cfunc`` creates a :keyword:`cdef` function.
* ``@cython.ccall`` creates a :keyword:`cpdef` function, i.e. one that Cython code
can call at the C level.
* ``@cython.locals`` declares local variables (see above). It can also be used to
declare types for the local variables that are used in the signature.
* ``@cython.inline`` is the equivalent of the C ``inline`` modifier.
Here is an example of a :keyword:`cdef` function::
@cython.cfunc
@cython.returns(cython.bint)
@cython.locals(a=cython.int, b=cython.int)
def c_compare(a,b):
return a == b
Further Cython functions and declarations
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* ``address`` is used in place of the ``&`` operator:: * ``address`` is used in place of the ``&`` operator::
cython.declare(x=cython.int, x_ptr=cython.p_int) cython.declare(x=cython.int, x_ptr=cython.p_int)
...@@ -55,7 +214,8 @@ The currently supported attributes of the ``cython`` module are: ...@@ -55,7 +214,8 @@ The currently supported attributes of the ``cython`` module are:
expressions.:: expressions.::
cython.declare(n=cython.longlong) cython.declare(n=cython.longlong)
print cython.sizeof(cython.longlong), cython.sizeof(n) print cython.sizeof(cython.longlong)
print cython.sizeof(n)
* ``struct`` can be used to create struct types.:: * ``struct`` can be used to create struct types.::
...@@ -71,51 +231,19 @@ The currently supported attributes of the ``cython`` module are: ...@@ -71,51 +231,19 @@ The currently supported attributes of the ``cython`` module are:
cdef MyStruct a cdef MyStruct a
* ``union`` creates union types with exactly the same syntax as ``struct`` * ``union`` creates union types with exactly the same syntax as ``struct``.
* ``typedef`` creates a new type:: * ``typedef`` creates a new type::
T = cython.typedef(cython.p_int) # ctypedef int* T T = cython.typedef(cython.p_int) # ctypedef int* T
* ``compiled`` is a special variable which is set to ``True`` when the compiler .. _magic_attributes_pxd:
runs, and ``False`` in the interpreter. Thus the code::
if cython.compiled:
print "Yep, I'm compiled."
else:
print "Just a lowly interpreted script."
will behave differently depending on whether or not the code is loaded as a
compiled :file:`.so` file or a plain :file:`.py` file.
Augmenting .pxd
---------------
If a :file:`.pxd` file is found with the same name as a :file:`.py` file, it will be
searched for :keyword:`cdef` classes and :keyword:`cdef`/:keyword:`cpdef`
functions and methods. It will then convert the corresponding
classes/functions/methods in the :file:`.py` file to be of the correct type. Thus if
one had :file:`a.pxd`::
cdef class A:
cpdef foo(self, int i)
the file :file:`a.py`:: Magic Attributes within the .pxd
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
class A:
def foo(self, i):
print "Big" if i > 1000 else "Small"
would be interpreted as::
cdef class A:
cpdef foo(self, int i):
print "Big" if i > 1000 else "Small"
The special Cython module can also be imported and used within the augmenting The special Cython module can also be imported and used within the augmenting
:file:`.pxd` file. This makes it possible to add types to a pure Python file without :file:`.pxd` file. For example, the following Python file :file:`dostuff.py`::
changing the file itself. For example, the following Python file
:file:`dostuff.py`::
def dostuff(n): def dostuff(n):
t = 0 t = 0
...@@ -123,7 +251,7 @@ changing the file itself. For example, the following Python file ...@@ -123,7 +251,7 @@ changing the file itself. For example, the following Python file
t += i t += i
return t return t
could be augmented with the following :file:`.pxd` file :file:`dostuff.pxd`:: can be augmented with the following :file:`.pxd` file :file:`dostuff.pxd`::
import cython import cython
...@@ -133,44 +261,3 @@ could be augmented with the following :file:`.pxd` file :file:`dostuff.pxd`:: ...@@ -133,44 +261,3 @@ could be augmented with the following :file:`.pxd` file :file:`dostuff.pxd`::
Besides the ``cython.locals`` decorator, the :func:`cython.declare` function can also be Besides the ``cython.locals`` decorator, the :func:`cython.declare` function can also be
used to add types to global variables in the augmenting :file:`.pxd` file. used to add types to global variables in the augmenting :file:`.pxd` file.
Note that normal Python (:keyword:`def`) functions cannot be declared in
:file:`.pxd` files, so it is currently impossible to override the types of
Python functions in :file:`.pxd` files if they use ``*args`` or ``**kwargs`` in their
signature, for instance.
Types
-----
There are numerous types built in to the Cython module. One has all the
standard C types, namely ``char``, ``short``, ``int``, ``long``, ``longlong``
as well as their unsigned versions ``uchar``, ``ushort``, ``uint``, ``ulong``,
``ulonglong``. One also has ``bint`` and ``Py_ssize_t``. For each type, there
are pointer types ``p_int``, ``pp_int``, . . ., up to three levels deep in
interpreted mode, and infinitely deep in compiled mode. The Python types int,
long and bool are interpreted as C ``int``, ``long`` and ``bint``
respectively. Also, the Python types ``list``, ``dict``, ``tuple``, . . . may
be used, as well as any user defined types.
Pointer types may be constructed with ``cython.pointer(cython.int)``, and
arrays as ``cython.int[10]``. A limited attempt is made to emulate these more
complex types, but only so much can be done from the Python language.
Extension types and cdef functions
-----------------------------------
Use the ``@cython.cclass`` decorator to create a ``cdef class``.
Use the ``@cython.cfunc`` decorator for :keyword:`cdef` functions
and the ``@cython.ccall`` decorators for :keyword:`cpdef` functions
respectively. To declare the argument types, use the
``@cython.locals()`` decorator. For the return type, use
``@cython.returns(a_type)``. The C ``inline`` modifier is available
as ``@cython.inline`` decorator.
Here is an example of a :keyword:`cdef` function::
@cython.cfunc
@cython.returns(cython.bint)
@cython.locals(a=cython.int, b=cython.int)
def c_compare(a,b):
return a == b
...@@ -31,7 +31,7 @@ and C :keyword:`struct`, :keyword:`union` or :keyword:`enum` types:: ...@@ -31,7 +31,7 @@ and C :keyword:`struct`, :keyword:`union` or :keyword:`enum` types::
float *eggs float *eggs
cdef enum CheeseType: cdef enum CheeseType:
cheddar, edam, cheddar, edam,
camembert camembert
cdef enum CheeseState: cdef enum CheeseState:
...@@ -64,6 +64,30 @@ an anonymous :keyword:`enum` declaration for this purpose, for example,:: ...@@ -64,6 +64,30 @@ an anonymous :keyword:`enum` declaration for this purpose, for example,::
ctypedef int* IntPtr ctypedef int* IntPtr
Types
-----
There are numerous types built in to the Cython module. It provides all the
standard C types, namely ``char``, ``short``, ``int``, ``long``, ``longlong``
as well as their unsigned versions ``uchar``, ``ushort``, ``uint``, ``ulong``,
``ulonglong``. One also has ``bint`` and ``Py_ssize_t``.
For each type, there are pointer types ``p_int``, ``pp_int``, . . ., up to
three levels deep in interpreted mode, and infinitely deep in compiled mode.
The Python types int, long and bool are interpreted as C ``int``, ``long``
and ``bint`` respectively.
Also, the Python types ``list``, ``dict``, ``tuple``, . . . may
be used, as well as any user defined types. However at the moment they add very
few in terms of optimization to a simple ``object`` typing.
Pointer types may be constructed with ``cython.pointer(cython.int)``, and
arrays as ``cython.int[10]``. A limited attempt is made to emulate these more
complex types, but only so much can be done from the Python language.
Grouping multiple C declarations Grouping multiple C declarations
-------------------------------- --------------------------------
...@@ -92,14 +116,14 @@ Python objects as parameters and return Python objects. ...@@ -92,14 +116,14 @@ Python objects as parameters and return Python objects.
C functions are defined using the new :keyword:`cdef` statement. They take C functions are defined using the new :keyword:`cdef` statement. They take
either Python objects or C values as parameters, and can return either Python either Python objects or C values as parameters, and can return either Python
objects or C values. objects or C values.
Within a Cython module, Python functions and C functions can call each other Within a Cython module, Python functions and C functions can call each other
freely, but only Python functions can be called from outside the module by freely, but only Python functions can be called from outside the module by
interpreted Python code. So, any functions that you want to "export" from your interpreted Python code. So, any functions that you want to "export" from your
Cython module must be declared as Python functions using def. Cython module must be declared as Python functions using def.
There is also a hybrid function, called :keyword:`cpdef`. A :keyword:`cpdef` There is also a hybrid function, called :keyword:`cpdef`. A :keyword:`cpdef`
can be called from anywhere, but uses the faster C calling conventions can be called from anywhere, but uses the faster C calling conventions
when being called from other Cython code. A :keyword:`cpdef` can also be overridden when being called from other Cython code. A :keyword:`cpdef` can also be overridden
by a Python method on a subclass or an instance attribute, even when called from Cython. by a Python method on a subclass or an instance attribute, even when called from Cython.
If this happens, most performance gains are of course lost and even if it does not, If this happens, most performance gains are of course lost and even if it does not,
...@@ -212,18 +236,18 @@ This form causes Cython to generate a call to :c:func:`PyErr_Occurred` after ...@@ -212,18 +236,18 @@ This form causes Cython to generate a call to :c:func:`PyErr_Occurred` after
every call to spam, regardless of what value it returns. If you have a every call to spam, regardless of what value it returns. If you have a
function returning void that needs to propagate errors, you will have to use function returning void that needs to propagate errors, you will have to use
this form, since there isn't any return value to test. this form, since there isn't any return value to test.
Otherwise there is little use for this form. Otherwise there is little use for this form.
An external C++ function that may raise an exception can be declared with:: An external C++ function that may raise an exception can be declared with::
cdef int spam() except + cdef int spam() except +
See :ref:`wrapping-cplusplus` for more details. See :ref:`wrapping-cplusplus` for more details.
Some things to note: Some things to note:
* Exception values can only declared for functions returning an integer, enum, * Exception values can only declared for functions returning an integer, enum,
float or pointer type, and the value must be a constant expression. float or pointer type, and the value must be a constant expression.
Void functions can only use the ``except *`` form. Void functions can only use the ``except *`` form.
* The exception value specification is part of the signature of the function. * The exception value specification is part of the signature of the function.
If you're passing a pointer to a function as a parameter or assigning it If you're passing a pointer to a function as a parameter or assigning it
...@@ -236,7 +260,7 @@ Some things to note: ...@@ -236,7 +260,7 @@ Some things to note:
* You don't need to (and shouldn't) declare exception values for functions * You don't need to (and shouldn't) declare exception values for functions
which return Python objects. Remember that a function with no declared which return Python objects. Remember that a function with no declared
return type implicitly returns a Python object. (Exceptions on such functions return type implicitly returns a Python object. (Exceptions on such functions
are implicitly propagated by returning NULL.) are implicitly propagated by returning NULL.)
Checking return values of non-Cython functions Checking return values of non-Cython functions
...@@ -260,7 +284,7 @@ return value and raise it yourself, for example,:: ...@@ -260,7 +284,7 @@ return value and raise it yourself, for example,::
if p == NULL: if p == NULL:
raise SpamError("Couldn't open the spam file") raise SpamError("Couldn't open the spam file")
Automatic type conversions Automatic type conversions
========================== ==========================
...@@ -287,7 +311,7 @@ possibilities. ...@@ -287,7 +311,7 @@ possibilities.
| struct | | dict | | struct | | dict |
+----------------------------+--------------------+------------------+ +----------------------------+--------------------+------------------+
.. [#] The conversion is to/from str for Python 2.x, and bytes for Python 3.x. .. [#] The conversion is to/from str for Python 2.x, and bytes for Python 3.x.
Caveats when using a Python string in a C context Caveats when using a Python string in a C context
------------------------------------------------- -------------------------------------------------
...@@ -352,9 +376,9 @@ direct equivalent in Python. ...@@ -352,9 +376,9 @@ direct equivalent in Python.
* An integer literal is treated as a C constant, and will * An integer literal is treated as a C constant, and will
be truncated to whatever size your C compiler thinks appropriate. be truncated to whatever size your C compiler thinks appropriate.
To get a Python integer (of arbitrary precision) cast immediately to To get a Python integer (of arbitrary precision) cast immediately to
an object (e.g. ``<object>100000000000000000000``). The ``L``, ``LL``, an object (e.g. ``<object>100000000000000000000``). The ``L``, ``LL``,
and ``U`` suffixes have the same meaning as in C. and ``U`` suffixes have the same meaning as in C.
* There is no ``->`` operator in Cython. Instead of ``p->x``, use ``p.x`` * There is no ``->`` operator in Cython. Instead of ``p->x``, use ``p.x``
* There is no unary ``*`` operator in Cython. Instead of ``*p``, use ``p[0]`` * There is no unary ``*`` operator in Cython. Instead of ``*p``, use ``p[0]``
* There is an ``&`` operator, with the same semantics as in C. * There is an ``&`` operator, with the same semantics as in C.
...@@ -495,7 +519,7 @@ Some things to note about the for-from loop: ...@@ -495,7 +519,7 @@ Some things to note about the for-from loop:
* The name between the lower and upper bounds must be the same as the target * The name between the lower and upper bounds must be the same as the target
name. name.
* The direction of iteration is determined by the relations. If they are both * The direction of iteration is determined by the relations. If they are both
from the set {``<``, ``<=``} then it is upwards; if they are both from the set from the set {``<``, ``<=``} then it is upwards; if they are both from the set
{``>``, ``>=``} then it is downwards. (Any other combination is disallowed.) {``>``, ``>=``} then it is downwards. (Any other combination is disallowed.)
Like other Python looping statements, break and continue may be used in the Like other Python looping statements, break and continue may be used in the
...@@ -505,8 +529,8 @@ body, and the loop may have an else clause. ...@@ -505,8 +529,8 @@ body, and the loop may have an else clause.
The include statement The include statement
===================== =====================
.. warning:: .. warning::
Historically the ``include`` statement was used for sharing declarations. Historically the ``include`` statement was used for sharing declarations.
Use :ref:`sharing-declarations` instead. Use :ref:`sharing-declarations` instead.
A Cython source file can include material from other files using the include A Cython source file can include material from other files using the include
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment