The sections in this page were moved to the :ref:`language-basics` in the userguide.
=================
Cython File Types
=================
.. NOW IN USER GUIDE, DO NOT TOUCH
There are three file types in Cython:
* Implementation files carry a ``.pyx`` suffix
* Definition files carry a ``.pxd`` suffix
* Include files which carry a ``.pxi`` suffix
This section was moved to :ref:`cython_file_types`.
Implementation File
===================
...
...
@@ -27,828 +21,158 @@ Implementation File
What can it contain?
--------------------
* Basically anything Cythonic, but see below.
What can't it contain?
----------------------
* There are some restrictions when it comes to **extension types**, if the extension type is
already defined else where... **more on this later**
Definition File
===============
What can it contain?
--------------------
* Any kind of C type declaration.
* ``extern`` C function or variable declarations.
* Declarations for module implementations.
* The definition parts of **extension types**.
* All declarations of functions, etc., for an **external library**
What can't it contain?
----------------------
* Any non-extern C variable declaration.
* Implementations of C or Python functions.
* Python class definitions
* Python executable statements.
* Any declaration that is defined as **public** to make it accessible to other Cython modules.
* This is not necessary, as it is automatic.
* a **public** declaration is only needed to make it accessible to **external C code**.
What else?
----------
cimport
```````
* Use the **cimport** statement, as you would Python's import statement, to access these files
from other definition or implementation files.
* **cimport** does not need to be called in ``.pyx`` file for ``.pxd`` file that has the
same name, as they are already in the same namespace.
* For cimport to find the stated definition file, the path to the file must be appended to the
``-I`` option of the **Cython compile command**.
compilation order
`````````````````
* When a ``.pyx`` file is to be compiled, Cython first checks to see if a corresponding ``.pxd`` file
exits and processes it first.
Include File
============
What can it contain?
--------------------
* Any Cythonic code really, because the entire file is textually embedded at the location
you prescribe.
How do I use it?
----------------
* Include the ``.pxi`` file with an ``include`` statement like: ``include "spamstuff.pxi``
* The ``include`` statement can appear anywhere in your Cython file and at any indentation level
* The code in the ``.pxi`` file needs to be rooted at the "zero" indentation level.
* The included code can itself contain other ``include`` statements.
====================
Declaring Data Types
====================
As a dynamic language, Python encourages a programming style of considering classes and objects in terms of their methods and attributes, more than where they fit into the class hierarchy.
This can make Python a very relaxed and comfortable language for rapid development, but with a price - the 'red tape' of managing data types is dumped onto the interpreter. At run time, the interpreter does a lot of work searching namespaces, fetching attributes and parsing argument and keyword tuples. This run-time ‘late binding’ is a major cause of Python’s relative slowness compared to ‘early binding’ languages such as C++.
However with Cython it is possible to gain significant speed-ups through the use of ‘early binding’ programming techniques.
.. note:: Typing is not a necessity
Providing static typing to parameters and variables is convenience to speed up your code, but it is not a necessity. Optimize where and when needed.
In fact, typing can *slow down* your code in the case where the
typing does not allow optimizations but where Cython still needs to
check that the type of some object matches the declared type.
This section was moved to :ref:`declaring_data_types`.
The cdef Statement
==================
The ``cdef`` statement is used to make C level declarations for:
:Variables:
::
cdef int i, j, k
cdef float f, g[42], *h
:Structs:
::
cdef struct Grail:
int age
float volume
.. note::
Structs can be declared as ``cdef packed struct``, which has
the same effect as the C directive ``#pragma pack(1)``.
:Unions:
::
cdef union Food:
char *spam
float *eggs
:Enums:
::
cdef enum CheeseType:
cheddar, edam,
camembert
Declaring an enum as ``cpdef`` will create a :pep:`435`-style Python wrapper::
cpdef enum CheeseState:
hard = 1
soft = 2
runny = 3
:Functions:
::
cdef int eggs(unsigned long l, float f):
...
:Extension Types:
::
cdef class Spam:
...
.. note:: Constants
Constants can be defined by using an anonymous enum::
cdef enum:
tons_of_spam = 3
This section was moved to :ref:`c_variable_and_type_definitions`.
Grouping cdef Declarations
==========================
A series of declarations can grouped into a ``cdef`` block::
cdef:
struct Spam:
int tons
int i
float f
Spam *p
void f(Spam *s):
print s.tons, "Tons of spam"
.. note:: ctypedef statement
The ``ctypedef`` statement is provided for naming types::
ctypedef unsigned long ULong
ctypedef int *IntPtr
This section was moved to :ref:`c_variable_and_type_definitions`.
.. _typing_types:
C types and Python classes
==========================
There are three kinds of types that you can declare:
1. C types, like ``cdef double x = 1.0``.
In the C code that Cython generates, this will create a C variable
of type ``double``. So working with this variable is exactly as fast
as working with a C variable of that type.
2. Builtin Python classes like ``cdef list L = []``.
This requires an *exact* match of the class, it does not allow
subclasses. This allows Cython to optimize code by accessing
internals of the builtin class.
Cython uses a C variable of type ``PyObject*``.
3. Extension types (declared with ``cdef class``).
This does allow subclasses. This typing is mostly used to access
``cdef`` methods and attributes of the extension type.
The C code uses a variable which is a pointer to a structure of the
specific type, something like ``struct MyExtensionTypeObject*``.
This section was moved to :ref:`types`.
Parameters
==========
* Both C and Python **function** types can be declared to have parameters with a given C data type.
* Use normal C declaration syntax::
def spam(int i, char *s):
...
cdef int eggs(unsigned long l, float f):
...
* As these parameters are passed into a Python declared function,
they are automatically **converted** to the specified C type value,
if a conversion is possible and safe. This applies to numeric and
string types, as well as some C++ container types.
* If no type is specified for a parameter or a return value, it is assumed to be a Python object.
* The following takes two Python objects as parameters and returns a Python object::
cdef spamobjs(x, y):
...
.. note::
This is different from the C language behavior, where missing types are assumed as ``int`` by default.
* Python object types have reference counting performed according to the standard Python/C-API rules:
* Borrowed references are taken as parameters
* New references are returned
.. warning::
This only applies to Cython code. Other Python packages which
are implemented in C like NumPy may not follow these conventions.
* The name ``object`` can be used to explicitly declare something as a Python Object.
* For sake of code clarity, it recommended to always use ``object`` explicitly in your code.
* This is also useful for cases where the name being declared would otherwise be taken for a type::
cdef foo(object int):
...
* As a return type::
cdef object foo(object int):
...
This section was moved to :ref:`python_functions_vs_c_functions`.
Automatic Type Conversion
=========================
* For basic numeric and string types, in most situations, when a Python object is used in the context of a C value and vice versa.
* The following table summarizes the conversion possibilities, assuming ``sizeof(int) == sizeof(long)``:
* A Python string, passed to C context expecting a ``char*``, is only valid as long as the Python string exists.
* A reference to the Python string must be kept around for as long as the C string is needed.
* If this can't be guaranteed, then make a copy of the C string.
* Cython may produce an error message: ``Obtaining char* from a temporary Python value`` and will not resume compiling in situations like this::
cdef char *s
s = pystring1 + pystring2
* The reason is that concatenating two strings in Python produces a temporary variable.
* The variable is decrefed, and the Python string deallocated as soon as the statement has finished,
* Therefore the lvalue **``s``** is left dangling.
* The solution is to assign the result of the concatenation to a Python variable, and then obtain the ``char*`` from that::
cdef char *s
p = pystring1 + pystring2
s = p
.. note::
**It is up to you to be aware of this, and not to depend on Cython's error message, as it is not guaranteed to be generated for every situation.**
This section was moved to :ref:`type-conversion`.
Type Casting
============
* The syntax used in type casting uses ``"<"`` and ``">"``, for example::
cdef char *p
cdef float *q
p = <char*>q
* If one of the types is a Python object for ``<type>x``, Cython will try to do a coercion.
.. note:: Cython will not stop a casting where there is no conversion, but it will emit a warning.
* To get the address of some Python object, use a cast to a pointer type
like ``<void*>`` or ``<PyObject*>``.
* The precedence of ``<...>`` is such that ``<type>a.b.c`` is interpreted as ``<type>(a.b.c)``.
This section was moved to :ref:`type_casting`.
Checked Type Casts
------------------
* A cast like ``<MyExtensionType>x`` will cast x to the class
``MyExtensionType`` without any checking at all.
* To have a cast checked, use the syntax like: ``<MyExtensionType?>x``.
In this case, Cython will apply a runtime check that raises a ``TypeError``
if ``x`` is not an instance of ``MyExtensionType``.
As explained in :ref:`typing_types`, this tests for the exact class
for builtin types, but allows subclasses for extension types.
This section was moved to :ref:`checked_type_casts`.
==========================
Statements and Expressions
==========================
* For the most part, control structures and expressions follow Python syntax.
* When applied to Python objects, the semantics are the same unless otherwise noted.
* Most Python operators can be applied to C values with the obvious semantics.
* An expression with mixed Python and C values will have **conversions** performed automatically.
* Python operations are automatically checked for errors, with the appropriate action taken.
This section was moved to :ref:`statements_and_expressions`.
Differences Between Cython and C
================================
* Most notable are C constructs which have no direct equivalent in Python.
* An integer literal is treated as a C constant
* It will be truncated to whatever size your C compiler thinks appropriate.
* Cast to a Python object like this::
<object>10000000000000000000
* The ``"L"``, ``"LL"`` and the ``"U"`` suffixes have the same meaning as in C
* There is no ``->`` operator in Cython.. instead of ``p->x``, use ``p.x``.
* There is no ``*`` operator in Cython.. instead of ``*p``, use ``p[0]``.
* ``&`` is permissible and has the same semantics as in C.
* ``NULL`` is the null C pointer.
* Do NOT use 0.
* ``NULL`` is a reserved word in Cython
* Syntax for **Type casts** are ``<type>value``.
Scope Rules
===========
* All determination of scoping (local, module, built-in) in Cython is determined statically.
* As with Python, a variable assignment which is not declared explicitly is implicitly declared to be a Python variable residing in the scope where it was assigned.
.. note::
* Module-level scope behaves the same way as a Python local scope if you refer to the variable before assigning to it.
* Tricks, like the following will NOT work in Cython::
try:
x = True
except NameError:
True = 1
* The above example will not work because ``True`` will always be looked up in the module-level scope. Do the following instead::
import __builtin__
try:
True = __builtin__.True
except AttributeError:
True = 1
Built-in Constants
==================
Predefined Python built-in constants:
* None
* True
* False
Operator Precedence
===================
* Cython uses Python precedence order, not C
For-loops
==========
The "for ... in iterable" loop works as in Python, but is even more versatile
in Cython as it can additionally be used on C types.
* ``range()`` is C optimized when the index value has been declared by ``cdef``,
for example::
cdef size_t i
for i in range(n):
...
* Iteration over C arrays and sliced pointers is supported and automatically
infers the type of the loop variable, e.g.::
cdef double* data = ...
for x in data[:10]:
...
* Iterating over many builtin types such as lists and tuples is optimized.
* There is also a more verbose C-style for-from syntax which, however, is
deprecated in favour of the normal Python "for ... in range()" loop. You
might still find it in legacy code that was written for Pyrex, though.
* The target expression must be a plain variable name.
* The name between the lower and upper bounds must be the same as the target name.
for i from 0 <= i < n:
...
* Or when using a step size::
for i from 0 <= i < n by s:
...
* To reverse the direction, reverse the conditional operation::
for i from n > i >= 0:
...
* The ``break`` and ``continue`` statements are permissible.
* Can contain an else clause.
=====================
Functions and Methods
=====================
* There are three types of function declarations in Cython as the sub-sections show below.
* Only "Python" functions can be called outside a Cython module from *Python interpreted code*.
This section was moved to :ref:`python_functions_vs_c_functions`.
Callable from Python (def)
==========================
* Are declared with the ``def`` statement
* Are called with Python objects
* Return Python objects
* See **Parameters** for special consideration
.. _cdef:
Callable from C (cdef)
======================
* Are declared with the ``cdef`` statement.
* Are called with either Python objects or C values.
* Can return either Python objects or C values.
.. _cpdef:
Callable from both Python and C (cpdef)
=======================================
* Are declared with the ``cpdef`` statement.
* Can be called from anywhere, because it uses a little Cython magic.
* Uses the faster C calling conventions when being called from other Cython code.
Overriding
==========
``cpdef`` methods can override ``cdef`` methods::
cdef class A:
cdef foo(self):
print "A"
cdef class B(A)
cdef foo(self, x=None)
print "B", x
cdef class C(B):
cpdef foo(self, x=True, int k=3)
print "C", x, k
When subclassing an extension type with a Python class,
``def`` methods can override ``cpdef`` methods but not ``cdef``
methods::
cdef class A:
cdef foo(self):
print("A")
cdef class B(A):
cpdef foo(self):
print("B")
class C(B): # NOTE: not cdef class
def foo(self):
print("C")
If ``C`` above would be an extension type (``cdef class``),
this would not work correctly.
The Cython compiler will give a warning in that case.
This section was moved to :ref:`overriding_in_extension_types`.
Function Pointers
=================
* Functions declared in a ``struct`` are automatically converted to function pointers.
* see **using exceptions with function pointers**
Python Built-ins
================
Cython compiles calls to most built-in functions into direct calls to
the corresponding Python/C API routines, making them particularly fast.
Only direct function calls using these names are optimised. If you do
something else with one of these names that assumes it's a Python object,
such as assign it to a Python variable, and later call it, the call will
Note 1: Pyrex originally provided a function :func:`getattr3(obj, name, default)`
corresponding to the three-argument form of the Python builtin :func:`getattr()`.
Cython still supports this function, but the usage is deprecated in favour of
the normal builtin, which Cython can optimise in both forms.
This section was moved to :ref:`built_in_functions`.
Optional Arguments
==================
* Are supported for ``cdef`` and ``cpdef`` functions
* There are differences though whether you declare them in a ``.pyx`` file or a ``.pxd`` file:
* When in a ``.pyx`` file, the signature is the same as it is in Python itself::
cdef class A:
cdef foo(self):
print "A"
cdef class B(A)
cdef foo(self, x=None)
print "B", x
cdef class C(B):
cpdef foo(self, x=True, int k=3)
print "C", x, k
* When in a ``.pxd`` file, the signature is different like this example: ``cdef foo(x=*)``::
cdef class A:
cdef foo(self)
cdef class B(A)
cdef foo(self, x=*)
cdef class C(B):
cpdef foo(self, x=*, int k=*)
* The number of arguments may increase when subclassing, but the arg types and order must be the same.
* There may be a slight performance penalty when the optional arg is overridden with one that does not have default values.
This section was moved to :ref:`optional_arguments`.
Keyword-only Arguments
=======================
* As in Python 3, ``def`` functions can have keyword-only arguments listed after a ``"*"`` parameter and before a ``"**"`` parameter if any::
def f(a, b, *args, c, d = 42, e, **kwds):
...
* Shown above, the ``c``, ``d`` and ``e`` arguments can not be passed as positional arguments and must be passed as keyword arguments.
* Furthermore, ``c`` and ``e`` are required keyword arguments since they do not have a default value.
* If the parameter name after the ``"*"`` is omitted, the function will not accept any extra positional arguments::
def g(a, b, *, c, d):
...
* Shown above, the signature takes exactly two positional parameters and has two required keyword parameters
This section was moved to :ref:`keyword_only_argument`.
============================
Error and Exception Handling
============================
* A plain ``cdef`` declared function, that does not return a Python object...
* Has no way of reporting a Python exception to it's caller.
* Will only print a warning message and the exception is ignored.
* In order to propagate exceptions like this to it's caller, you need to declare an exception value for it.
* There are three forms of declaring an exception for a C compiled program.
* First::
cdef int spam() except -1:
...
* In the example above, if an error occurs inside spam, it will immediately return with the value of ``-1``, causing an exception to be propagated to it's caller.
* Functions declared with an exception value, should explicitly prevent a return of that value.
* Second::
cdef int spam() except? -1:
...
* Used when a ``-1`` may possibly be returned and is not to be considered an error.
* The ``"?"`` tells Cython that ``-1`` only indicates a *possible* error.
* Now, each time ``-1`` is returned, Cython generates a call to ``PyErr_Occurred`` to verify it is an actual error.
* Third::
cdef int spam() except *
* A call to ``PyErr_Occurred`` happens *every* time the function gets called.
.. note:: Returning ``void``
A need to propagate errors when returning ``void`` must use this version.
* Exception values can only be declared for functions returning an..
* integer
* enum
* float
* pointer type
* Must be a constant expression
.. note::
.. note:: Function pointers
* Require the same exception value specification as it's user has declared.
* Use cases here are when used as parameters and when assigned to a variable::
int (*grail)(int, char *) except -1
.. note:: Python Objects
* Declared exception values are **not** need.
* Remember that Cython assumes that a function without a declared return value, returns a Python object.
* Exceptions on such functions are implicitly propagated by returning ``NULL``
.. note:: C++
* For exceptions from C++ compiled programs, see **Wrapping C++ Classes**
This section was moved to :ref:`error_return_values`.
Checking return values for non-Cython functions..
=================================================
* Do not try to raise exceptions by returning the specified value.. Example::
