Skip to content

C
cython
Commit cd42e9bf authored by scoder's avatar scoder Committed by GitHub
Browse files
Options

Merge pull request #2284 from gabrieldemarmiesse/replace_text_with_links_ext_types 

Replaced the text in reference/extension_types.rst by links to the corresponding sections in the userguide.
parents 78f4d7b8 26c677d5
Hide whitespace changes
Inline Side-by-side
Showing with 51 additions and 469 deletions
docs/src/reference/extension_types.rst
View file @ cd42e9bf
.. highlight:: cython .. highlight:: cython
.. _extension_types:
*************** ***************
Extension Types Extension Types
*************** ***************
* Normal Python as well as extension type classes can be defined. .. note::
* Extension types:
* Are considered by Python as "built-in" types.
* Can be used to wrap arbitrary C-data structures, and provide a Python-like interface to them from Python.
* Attributes and methods can be called from Python or Cython code
* Are defined by the ``cdef class`` statement.
::
cdef class Shrubbery:
cdef int width, height
def __init__(self, w, h):
self.width = w
self.height = h
def describe(self): The sections in this page were moved to the :ref:`extension-types`
print "This shrubbery is", self.width, \ and :ref:`special-methods` in the userguide.
"by", self.height, "cubits."
========== ==========
Attributes Attributes
========== ==========
* Are stored directly in the object's C struct. This section was moved to :ref:`readonly`.
* Are fixed at compile time.
* You can't add attributes to an extension type instance at run time like in normal Python, unless you define a ``__dict__`` attribute.
* You can sub-class the extension type in Python to add attributes at run-time.
* There are two ways to access extension type attributes:
* By Python look-up.
* Python code's only method of access.
* By direct access to the C struct from Cython code.
* Cython code can use either method of access, though.
* By default, extension type attributes are:
* Only accessible by direct access.
* Not accessible from Python code.
* To make attributes accessible to Python, they must be declared ``public`` or ``readonly``::
cdef class Shrubbery:
cdef public int width, height
cdef readonly float depth
* The ``width`` and ``height`` attributes are readable and writable from Python code.
* The ``depth`` attribute is readable but not writable.
.. note::
.. note::
You can only expose simple C types, such as ints, floats, and strings, for Python access. You can also expose Python-valued attributes.
.. note::
The ``public`` and ``readonly`` options apply only to Python access, not direct access. All the attributes of an extension type are always readable and writable by C-level access.
======= =======
Methods Methods
======= =======
* ``self`` is used in extension type methods just like it normally is in Python.
* See **Functions and Methods**; all of which applies here.
========== ==========
Properties Properties
========== ==========
* Cython provides a special (deprecated) syntax:: This section was moved to :ref:`properties`.
cdef class Spam:
property cheese:
"A doc string can go here."
def __get__(self):
# This is called when the property is read.
...
def __set__(self, value):
# This is called when the property is written.
...
def __del__(self):
# This is called when the property is deleted.
* The ``__get__()``, ``__set__()``, and ``__del__()`` methods are all optional.
* If they are omitted, an exception is raised on attribute access.
* Below, is a full example that defines a property which can..
* Add to a list each time it is written to (``"__set__"``).
* Return the list when it is read (``"__get__"``).
* Empty the list when it is deleted (``"__del__"``).
::
# cheesy.pyx
cdef class CheeseShop:
cdef object cheeses
def __cinit__(self):
self.cheeses = []
property cheese: # note that this syntax is deprecated
def __get__(self):
return "We don't have: %s" % self.cheeses
def __set__(self, value):
self.cheeses.append(value)
def __del__(self):
del self.cheeses[:]
# Test input
from cheesy import CheeseShop
shop = CheeseShop()
print shop.cheese
shop.cheese = "camembert"
print shop.cheese
shop.cheese = "cheddar"
print shop.cheese
del shop.cheese
print shop.cheese
::
# Test output
We don't have: []
We don't have: ['camembert']
We don't have: ['camembert', 'cheddar']
We don't have: []
=============== ===============
Special Methods Special Methods
=============== ===============
.. note:: This section was moved to :ref:`special-methods`.
#. The semantics of Cython's special methods are similar in principle to that of Python's.
#. There are substantial differences in some behavior.
#. Some Cython special methods have no Python counter-part.
* See the :ref:`special_methods_table` for the many that are available.
Declaration Declaration
=========== ===========
* Must be declared with ``def`` and cannot be declared with ``cdef``. This section was moved to :ref:`declaration`.
* Performance is not affected by the ``def`` declaration because of special calling conventions
Docstrings Docstrings
========== ==========
* Docstrings are not supported yet for some special method types. This section was moved to :ref:`docstrings`.
* They can be included in the source, but may not appear in the corresponding ``__doc__`` attribute at run-time.
* This a Python library limitation because the ``PyTypeObject`` data structure is limited
Initialization: ``__cinit__()`` and ``__init__()`` Initialization: ``__cinit__()`` and ``__init__()``
================================================== ==================================================
* The object initialisation follows (mainly) three steps: This section was moved to :ref:`initialisation_methods`.
* (Internal) allocation, recursively going from subclass to base class.
* Low-level initialisation along the way back, calling ``__cinit__()`` at each level.
* Python initialisation, explicitly calling ``__init__()`` recursively from subclass to base class.
* Any arguments passed to the extension type's constructor
will be passed to both initialization methods.
* ``__cinit__()`` is where you should perform C-level initialization of the object
* This includes any allocation of C data structures.
* **Caution** is warranted as to what you do in this method.
* The object may not be a fully valid Python object when it is called.
* Calling Python objects, including the extensions own methods, may be hazardous.
* By the time ``__cinit__()`` is called...
* Memory has been allocated for the object.
* All C-level attributes have been initialized to 0 or null.
* The ``__cinit__()`` methods of all base types have been called, starting with the top-most one.
* Subtypes are not fully initialised yet.
* Python object attributes of the type itself have been initialized to ``None``.
* This initialization method is guaranteed to be called exactly once.
* For Extensions types that inherit a base type:
* The ``__cinit__()`` method of the base type is automatically called before this one.
* The inherited ``__cinit__()`` method cannot be called explicitly.
* Passing modified argument lists to the base type must be done through ``__init__()``.
* It may be wise to give the ``__cinit__()`` method both ``"*"`` and ``"**"`` arguments.
* Allows the method to accept or ignore additional arguments.
* Eliminates the need for a Python level sub-class, that changes the ``__init__()``
method's signature, to have to override both the ``__new__()`` and ``__init__()`` methods.
* If ``__cinit__()`` is declared to take no arguments except ``self``, it will ignore any
extra arguments passed to the constructor without complaining about a signature mis-match.
* ``__init__()`` is for higher-level initialization and is safer for Python access.
* By the time this method is called, the extension type is a fully valid Python object.
* All operations are safe.
* This method may sometimes be called more than once, or possibly not at all.
* Take this into consideration to make sure the design of your other methods are robust of this fact.
Note that all constructor arguments will be passed as Python objects.
This implies that non-convertible C types such as pointers or C++ objects
cannot be passed into the constructor from Cython code. If this is needed,
use a factory function instead that handles the object initialisation.
It often helps to directly call ``__new__()`` in this function to bypass the
call to the ``__init__()`` constructor.
Finalization: ``__dealloc__()`` Finalization: ``__dealloc__()``
=============================== ===============================
* This method is the counter-part to ``__cinit__()``. This section was moved to :ref:`finalization_method`.
* Any C-data that was explicitly allocated in the ``__cinit__()`` method should be freed here.
* Use caution in this method:
* The Python object to which this method belongs may not be completely intact at this point.
* Avoid invoking any Python operations that may touch the object.
* Don't call any of this object's methods.
* It's best to just deallocate C-data structures here.
* All Python attributes of your extension type object are deallocated by Cython after the ``__dealloc__()`` method returns.
Arithmetic Methods Arithmetic Methods
================== ==================
.. note:: Most of these methods behave differently than in Python This section was moved to :ref:`arithmetic_methods`.
* There are not "reversed" versions of these methods... there is no __radd__() for instance.
* If the first operand cannot perform the operation, the same method of the second operand is called, with the operands in the same order.
* Do not rely on the first parameter of these methods, being ``"self"`` or the right type.
* The types of both operands should be tested before deciding what to do.
* Return ``NotImplemented`` for unhandled, mis-matched operand types.
* The previously mentioned points..
* Also apply to 'in-place' method ``__ipow__()``.
* Do not apply to other 'in-place' methods like ``__iadd__()``, in that these always take ``self`` as the first argument.
Rich Comparisons Rich Comparisons
================ ================
There are two ways to implement comparison methods. This section was moved to :ref:`righ_comparisons`.
Depending on the application, one way or the other may be better:
* The first way uses the 6 Python
`special methods <https://docs.python.org/3/reference/datamodel.html#basic-customization>`_
``__eq__``, ``__lt__``, etc.
This is new since Cython 0.27 and works exactly as in plain Python classes.
* The second way uses a single special method ``__richcmp__``.
This implements all rich comparison operations in one method.
The signature is ``def __richcmp__(self, other, int op)`` matching the
`PyObject_RichCompare() <https://docs.python.org/3/c-api/object.html#c.PyObject_RichCompare>`_
Python/C API function.
The integer argument ``op`` indicates which operation is to be performed
as shown in the table below:
+-----+-----+-------+
| < | 0 | Py_LT |
+-----+-----+-------+
| == | 2 | Py_EQ |
+-----+-----+-------+
| > | 4 | Py_GT |
+-----+-----+-------+
| <= | 1 | Py_LE |
+-----+-----+-------+
| != | 3 | Py_NE |
+-----+-----+-------+
| >= | 5 | Py_GE |
+-----+-----+-------+
The named constants can be cimported from the ``cpython.object`` module.
They should generally be preferred over plain integers to improve readability.
The ``__next__()`` Method The ``__next__()`` Method
========================= =========================
* Extension types used to expose an iterator interface should define a ``__next__()`` method. This section was moved to :ref:`the__next__method`.
* **Do not** explicitly supply a ``next()`` method, because Python does that for you automatically.
=========== ===========
Subclassing Subclassing
=========== ===========
* An extension type may inherit from a built-in type or another extension type:: This section was moved to :ref:`subclassing`.
cdef class Parrot:
...
cdef class Norwegian(Parrot):
...
* A complete definition of the base type must be available to Cython
* If the base type is a built-in type, it must have been previously declared as an ``extern`` extension type.
* ``cimport`` can be used to import the base type, if the extern declared base type is in a ``.pxd`` definition file.
* In Cython, multiple inheritance is not permitted.. singular inheritance only
* Cython extension types can also be sub-classed in Python.
* Here multiple inheritance is permissible as is normal for Python.
* Even multiple extension types may be inherited, but C-layout of all the base classes must be compatible.
==================== ====================
Forward Declarations Forward Declarations
==================== ====================
* Extension types can be "forward-declared". This section was moved to :ref:`forward_declaring_extension_types`.
* This is necessary when two extension types refer to each other::
cdef class Shrubbery # forward declaration
cdef class Shrubber:
cdef Shrubbery work_in_progress
cdef class Shrubbery:
cdef Shrubber creator
* An extension type that has a base-class, requires that both forward-declarations be specified::
cdef class A(B)
...
cdef class A(B):
# attributes and methods
======================== ========================
Extension Types and None Extension Types and None
======================== ========================
* Parameters and C-variables declared as an Extension type, may take the value of ``None``. This section was moved to :ref:`extension_types_and_none`.
* This is analogous to the way a C-pointer can take the value of ``NULL``.
.. note::
#. Exercise caution when using ``None``
#. Read this section carefully.
* There is no problem as long as you are performing Python operations on it.
* This is because full dynamic type checking is applied
* When accessing an extension type's C-attributes, **make sure** it is not ``None``.
* Cython does not check this for reasons of efficiency.
* Be very aware of exposing Python functions that take extension types as arguments::
def widen_shrubbery(Shrubbery sh, extra_width): # This is dangerous
sh.width = sh.width + extra_width
* Users could **crash** the program by passing ``None`` for the ``sh`` parameter.
* This could be avoided by::
def widen_shrubbery(Shrubbery sh, extra_width):
if sh is None:
raise TypeError
sh.width = sh.width + extra_width
* Cython provides a more convenient way with a ``not None`` clause::
def widen_shrubbery(Shrubbery sh not None, extra_width):
sh.width = sh.width + extra_width
* Now this function automatically checks that ``sh`` is not ``None``, as well as that is the right type.
* ``not None`` can only be used in Python functions (declared with ``def`` **not** ``cdef``).
* For ``cdef`` functions, you will have to provide the check yourself.
* The ``self`` parameter of an extension type is guaranteed to **never** be ``None``.
* When comparing a value ``x`` with ``None``, and ``x`` is a Python object, note the following:
* ``x is None`` and ``x is not None`` are very efficient.
* They translate directly to C-pointer comparisons.
* ``x == None`` and ``x != None`` or ``if x: ...`` (a boolean condition), will invoke Python operations and will therefore be much slower.
================ ================
Weak Referencing Weak Referencing
================ ================
* By default, weak references are not supported. This section was moved to :ref:`making_extension_types_weak_referenceable`.
* It can be enabled by declaring a C attribute of the ``object`` type called ``__weakref__()``::
cdef class ExplodingAnimal:
"""This animal will self-destruct when it is
no longer strongly referenced."""
cdef object __weakref__
================== ==================
Dynamic Attributes Dynamic Attributes
================== ==================
* By default, you cannot dynamically add attributes to a ``cdef class`` instance at runtime. This section was moved to :ref:`dynamic_attributes`.
* It can be enabled by declaring a C attribute of the ``dict`` type called ``__dict__``::
cdef class ExtendableAnimal:
"""This animal can be extended with new
attributes at runtime."""
cdef dict __dict__
.. note::
#. This can have a performance penalty, especially when using ``cpdef`` methods in a class.
========================= =========================
External and Public Types External and Public Types
...@@ -453,118 +104,20 @@ External and Public Types ...@@ -453,118 +104,20 @@ External and Public Types
Public Public
====== ======
* When an extension type is declared ``public``, Cython will generate a C-header (".h") file. This section was moved to :ref:`public`.
* The header file will contain the declarations for it's **object-struct** and it's **type-object**.
* External C-code can now access the attributes of the extension type.
External External
======== ========
* An ``extern`` extension type allows you to gain access to the internals of: This section was moved to :ref:`external_extension_types`.
* Python objects defined in the Python core.
* Non-Cython extension modules
* The following example lets you get at the C-level members of Python's built-in "complex" object::
cdef extern from "complexobject.h":
struct Py_complex:
double real
double imag
ctypedef class __builtin__.complex [object PyComplexObject]:
cdef Py_complex cval
# A function which uses the above type
def spam(complex c):
print "Real:", c.cval.real
print "Imag:", c.cval.imag
.. note:: Some important things in the example:
#. ``ctypedef`` has been used because Python's header file has the struct declared with::
ctypedef struct {
...
} PyComplexObject;
#. The module of where this type object can be found is specified along side the name of the extension type. See **Implicit Importing**.
#. When declaring an external extension type...
* Don't declare any methods, because they are Python method class the are not needed.
* Similar to **structs** and **unions**, extension classes declared inside a ``cdef extern from`` block only need to declare the C members which you will actually need to access in your module.
Name Specification Clause Name Specification Clause
========================= =========================
.. note:: Only available to **public** and **extern** extension types. This section was moved to :ref:`name_specification_clause`.
* Example::
[object object_struct_name, type type_object_name ]
* ``object_struct_name`` is the name to assume for the type's C-struct.
* ``type_object_name`` is the name to assume for the type's statically declared type-object.
* The object and type clauses can be written in any order.
* For ``cdef extern from`` declarations, This clause **is required**.
* The object clause is required because Cython must generate code that is compatible with the declarations in the header file.
* Otherwise the object clause is optional.
* For public extension types, both the object and type clauses **are required** for Cython to generate code that is compatible with external C-code.
================================ ================================
Type Names vs. Constructor Names Type Names vs. Constructor Names
================================ ================================
* In a Cython module, the name of an extension type serves two distinct purposes: This section was moved to :ref:`types_names_vs_constructor_names`.
#. When used in an expression, it refers to a "module-level" global variable holding the type's constructor (i.e. it's type-object)
#. It can also be used as a C-type name to declare a "type" for variables, arguments, and return values.
* Example::
cdef extern class MyModule.Spam:
...
* The name "Spam" serves both of these roles.
* Only "Spam" can be used as the type-name.
* The constructor can be referred to by other names.
* Upon an explicit import of "MyModule"...
* ``MyModule.Spam()`` could be used as the constructor call.
* ``MyModule.Spam`` could not be used as a type-name
* When an "as" clause is used, the name specified takes over both roles::
cdef extern class MyModule.Spam as Yummy:
...
* ``Yummy`` becomes both type-name and a name for the constructor.
* There other ways of course, to get hold of the constructor, but ``Yummy`` is the only usable type-name.
docs/src/userguide/extension_types.rst
View file @ cd42e9bf
...@@ -194,6 +194,7 @@ The old behavior is always available by passing a tuple as the second parameter: ...@@ -194,6 +194,7 @@ The old behavior is always available by passing a tuple as the second parameter:
print isinstance(sh, Shrubbery) # Check the type of sh print isinstance(sh, Shrubbery) # Check the type of sh
print isinstance(sh, (Shrubbery,)) # Check sh.__class__ print isinstance(sh, (Shrubbery,)) # Check sh.__class__
.. _extension_types_and_none:
Extension types and None Extension types and None
========================= =========================
...@@ -263,6 +264,8 @@ counterparts. There is a :ref:`separate page <special-methods>` devoted to this ...@@ -263,6 +264,8 @@ counterparts. There is a :ref:`separate page <special-methods>` devoted to this
read it carefully before attempting to use any special methods in your read it carefully before attempting to use any special methods in your
extension types. extension types.
.. _properties:
Properties Properties
============ ============
...@@ -356,6 +359,8 @@ when it is deleted.:: ...@@ -356,6 +359,8 @@ when it is deleted.::
We don't have: ['camembert', 'cheddar'] We don't have: ['camembert', 'cheddar']
We don't have: [] We don't have: []
.. _subclassing:
Subclassing Subclassing
============= =============
...@@ -462,6 +467,7 @@ compatible types.:: ...@@ -462,6 +467,7 @@ compatible types.::
p.ptr = ptr p.ptr = ptr
return p return p
.. _forward_declaring_extension_types:
Forward-declaring extension types Forward-declaring extension types
=================================== ===================================
...@@ -526,6 +532,7 @@ statically sized freelist of ``N`` instances for a given type. Example:: ...@@ -526,6 +532,7 @@ statically sized freelist of ``N`` instances for a given type. Example::
penguin = None penguin = None
penguin = Penguin('fish 2') # does not need to allocate memory! penguin = Penguin('fish 2') # does not need to allocate memory!
.. _making_extension_types_weak_referenceable:
Making extension types weak-referenceable Making extension types weak-referenceable
========================================== ==========================================
...@@ -618,6 +625,8 @@ declaration makes an extension type defined in external C code available to a ...@@ -618,6 +625,8 @@ declaration makes an extension type defined in external C code available to a
Cython module. A public extension type declaration makes an extension type Cython module. A public extension type declaration makes an extension type
defined in a Cython module available to external C code. defined in a Cython module available to external C code.
.. _external_extension_types:
External extension types External extension types
------------------------ ------------------------
...@@ -673,6 +682,8 @@ built-in complex object.:: ...@@ -673,6 +682,8 @@ built-in complex object.::
declaration is inside a :keyword:`cdef` extern from block, you only need to declaration is inside a :keyword:`cdef` extern from block, you only need to
declare those C members which you wish to access. declare those C members which you wish to access.
.. _name_specification_clause:
Name specification clause Name specification clause
------------------------- -------------------------
...@@ -728,6 +739,8 @@ which corresponds to the implicit import statement:: ...@@ -728,6 +739,8 @@ which corresponds to the implicit import statement::
from My.Nested.Package import Spam as Yummy from My.Nested.Package import Spam as Yummy
.. _types_names_vs_constructor_names:
Type names vs. constructor names Type names vs. constructor names
-------------------------------- --------------------------------
...@@ -758,6 +771,8 @@ then Yummy becomes both the type name and a name for the constructor. Again, ...@@ -758,6 +771,8 @@ then Yummy becomes both the type name and a name for the constructor. Again,
there are other ways that you could get hold of the constructor, but only there are other ways that you could get hold of the constructor, but only
Yummy is usable as a type name. Yummy is usable as a type name.
.. _public:
Public extension types Public extension types
====================== ======================
......
docs/src/userguide/special_methods.rst
View file @ cd42e9bf
...@@ -13,12 +13,16 @@ mention. ...@@ -13,12 +13,16 @@ mention.
with the :keyword:`cdef class` statement. It doesn't apply to classes defined with the with the :keyword:`cdef class` statement. It doesn't apply to classes defined with the
Python :keyword:`class` statement, where the normal Python rules apply. Python :keyword:`class` statement, where the normal Python rules apply.
.. _declaration:
Declaration Declaration
------------ ------------
Special methods of extension types must be declared with :keyword:`def`, not Special methods of extension types must be declared with :keyword:`def`, not
:keyword:`cdef`. This does not impact their performance--Python uses different :keyword:`cdef`. This does not impact their performance--Python uses different
calling conventions to invoke these special methods. calling conventions to invoke these special methods.
.. _docstrings:
Docstrings Docstrings
----------- -----------
...@@ -28,6 +32,8 @@ won't show up in the corresponding :attr:`__doc__` attribute at run time. (This ...@@ -28,6 +32,8 @@ won't show up in the corresponding :attr:`__doc__` attribute at run time. (This
seems to be is a Python limitation -- there's nowhere in the `PyTypeObject` seems to be is a Python limitation -- there's nowhere in the `PyTypeObject`
data structure to put such docstrings.) data structure to put such docstrings.)
.. _initialisation_methods:
Initialisation methods: :meth:`__cinit__` and :meth:`__init__` Initialisation methods: :meth:`__cinit__` and :meth:`__init__`
--------------------------------------------------------------- ---------------------------------------------------------------
There are two methods concerned with initialising the object. There are two methods concerned with initialising the object.
...@@ -90,6 +96,8 @@ complaining about the signature mismatch. ...@@ -90,6 +96,8 @@ complaining about the signature mismatch.
.. [#] http://docs.python.org/reference/datamodel.html#object.__new__ .. [#] http://docs.python.org/reference/datamodel.html#object.__new__
.. _finalization_method:
Finalization method: :meth:`__dealloc__` Finalization method: :meth:`__dealloc__`
---------------------------------------- ----------------------------------------
...@@ -117,6 +125,8 @@ executed unless they are explicitly called by the subclass. ...@@ -117,6 +125,8 @@ executed unless they are explicitly called by the subclass.
.. Note:: There is no :meth:`__del__` method for extension types. .. Note:: There is no :meth:`__del__` method for extension types.
.. _arithmetic_methods:
Arithmetic methods Arithmetic methods
------------------- -------------------
...@@ -135,6 +145,8 @@ This also applies to the in-place arithmetic method :meth:`__ipow__`. It doesn't ...@@ -135,6 +145,8 @@ This also applies to the in-place arithmetic method :meth:`__ipow__`. It doesn't
to any of the other in-place methods (:meth:`__iadd__`, etc.) which always to any of the other in-place methods (:meth:`__iadd__`, etc.) which always
take `self` as the first argument. take `self` as the first argument.
.. _righ_comparisons:
Rich comparisons Rich comparisons
----------------- -----------------
...@@ -167,6 +179,8 @@ Depending on the application, one way or the other may be better: ...@@ -167,6 +179,8 @@ Depending on the application, one way or the other may be better:
These constants can be cimported from the ``cpython.object`` module. These constants can be cimported from the ``cpython.object`` module.
.. _the__next__method:
The :meth:`__next__` method The :meth:`__next__` method
---------------------------- ----------------------------
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7