Started tutorial, begun refactoring the compilation section, and updating the limitations.

conf.py

@@ -40,9 +40,9 @@ copyright = '2008, Stefan Behnel, Robert Bradshaw, William Stein, Gary Furnish,
 # other places throughout the built documents.
 #
 # The short X.Y version.
-version = '0.9.6'
+version = '0.9.8.1'
 # The full version, including alpha/beta/rc tags.
-release = '0.9.6.13.1'
+release = '0.9.8.1'
 
 # There are two options for replacing |today|: either, you set today to some
 # non-false value, then it is used:

docs/extension_types.rst

@@ -107,8 +107,8 @@ The same consideration applies to local variables, for example,::
         cdef Shrubbery sh2
         sh2 = Shrubbery()
         sh2.width = sh1.width
-    sh2.height = sh1.height
-    return sh2
+        sh2.height = sh1.height
+        return sh2
 
 Extension types and None
 ------------------------

docs/limitations.rst

 .. _cython-limitations-label:
 
+*************
 Limitations
-===========
+*************
 
-Unsupported Python features
-^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Unsupported Python Features
+============================
 
-Cython is not quite a full superset of Python. The following restrictions apply:
+One of our goals is to make Cython as compatible as possible with standard
+Python. This page lists the things that work in Python but not in Cython.
 
-* Function definitions (whether using ``def`` or ``cdef``) cannot be nested within
-  other function definitions.  
-* Class definitions can only appear at the top level of a module, 
-  not inside a function.
-* The ``import *`` form of import is not allowed anywhere (other forms of the
-  import statement are fine, though).  
-* Generators cannot be defined in Cython.
-* The ``globals()`` and ``locals()`` functions cannot be used.
+.. TODO: this limitation seems to be removed
+.. ::
 
-The above restrictions will most likely remain, since removing them would be
-difficult and they're not really needed for Cython's intended applications.
+..    from module import *
 
-There are also some temporary limitations, which may eventually be lifted, including:
+.. This relies on at-runtime insertion of objects into the current namespace and
+.. probably will be one of the few features never implemented (as any
+.. implementation would be very slow). However, there is the --pre-import option
+.. with treats all un-declared names as coming from the specified module, which
+.. has the same effect as putting "from module import *" at the top-level of the
+.. code. Note: the one difference is that builtins cannot be overriden in this
+.. way, as the 'pre-import' scope is even higher than the builtin scope.
 
+Nested def statements
+----------------------
+Function definitions (whether using ``def`` or ``cdef``) cannot be nested within
+other function definitions. ::
+
+    def make_func():
+        def f(x):
+            return x*x
+        return f
+
+(work in progress) This relies on functional closures
+
+Generators
+-----------
+
+Using the yield keywords. (work in progress) This relies on functional closures
+
+
+.. TODO Not really a limitation, rather an enchancement proposal
+
+.. Support for builtin types
+.. --------------------------
+
+.. Support for statically declaring types such as list and dict and sequence
+.. should be provided, and optimized code produced.
+
+.. This needs to be well thought-out, and I think Pyrex has some plans along
+.. these lines as well.
+
+Modulo '%' operation on floats
+-------------------------------
+::
+
+    a = b%c
+
+where `b` and `c` are floats will raise the error "Invalid operand types for '%' (float; float)"
+
+This can currently be worked around by putting::
+
+    cdef extern from "math.h":
+        double fmod(double x, double y)
+
+somewhere is the source file and then using::
+
+    a = fmod(b,c)
+
+
+Other Current Limitations
+==========================
+
+* The :func:`globals` and :func:`locals` functions cannot be used.
 * Class and function definitions cannot be placed inside control structures.
-* There is no support for Unicode.
 * Special methods of extension types cannot have functioning docstrings.
 * The use of string literals as comments is not recommended at present,
   because Cython doesn't optimize them away, and won't even accept them in places
@@ -38,7 +89,7 @@ Behaviour of class scopes
 In Python, referring to a method of a class inside the class definition, i.e.
 while the class is being defined, yields a plain function object, but in
 Cython it yields an unbound method [#]_. A consequence of this is that the
-usual idiom for using the ``classmethod`` and ``staticmethod`` functions,
+usual idiom for using the :func:`classmethod` and :func:`staticmethod` functions,
 e.g.::
 
     class Spam:
@@ -62,8 +113,8 @@ outside the class, and then assigning the result of ``classmethod`` or
 .. rubric:: Footnotes
 
 .. [#] The reason for the different behaviour of class scopes is that
-       Cython-defined Python functions are PyCFunction objects, not PyFunction
-       objects, and are not recognised by the machinery that creates a bound
-       or unbound method when a function is extracted from a class. To get
+       Cython-defined Python functions are ``PyCFunction`` objects, not
+       ``PyFunction`` objects, and are not recognised by the machinery that creates a
+       bound or unbound method when a function is extracted from a class. To get
        around this, Cython wraps each method in an unbound method object itself
        before storing it in the class's dictionary.

docs/overview.rst

@@ -57,94 +57,6 @@ behind the scenes, just as it is in interpreted Python code. And what's more,
 Cython lets you define new built-in Python types just as easily as you can
 define new classes in Python.
 
-Sound too good to be true? Read on and find out how it's done.
-
-The Basics of Cython
-====================
-
-The fundamental nature of Cython can be summed up as follows: Cython is Python
-with C data types.
-
-Cython is Python: Almost any piece of Python code is also valid Cython code.
-(There are a few :ref:`cython-limitations-label`, but this approximation will
-serve for now.) The Cython compiler will convert it into C code which makes
-equivalent calls to the Python/C API. 
-
-But Cython is much more than that, because parameters and variables can be
-declared to have C data types. Code which manipulates Python values and C
-values can be freely intermixed, with conversions occurring automatically
-wherever possible. Reference count maintenance and error checking of Python
-operations is also automatic, and the full power of Python's exception
-handling facilities, including the try-except and try-finally statements, is
-available to you -- even in the midst of manipulating C data.
-
-Here's a small example showing some of what can be done. It's a routine for
-finding prime numbers. You tell it how many primes you want, and it returns
-them as a Python list.
-
-:file:`primes.pyx`:
-
-.. code-block:: none 
-    :linenos:
-
-    def primes(int kmax):
-        cdef int n, k, i
-        cdef int p[1000]
-        result = []
-        if kmax > 1000:
-            kmax = 1000
-        k = 0
-        n = 2
-        while k < kmax:
-            i = 0
-            while i < k and n % p[i] != 0:
-                i = i + 1
-            if i == k:
-               p[k] = n
-               k = k + 1
-               result.append(n)
-            n = n + 1
-        return result
-
-You'll see that it starts out just like a normal Python function definition,
-except that the parameter ``kmax`` is declared to be of type ``int`` . This
-means that the object passed will be converted to a C integer (or a
-``TypeError.`` will be raised if it can't be).
-
-Lines 2 and 3 use the ``cdef`` statement to define some local C variables.
-Line 4 creates a Python list which will be used to return the result. You'll
-notice that this is done exactly the same way it would be in Python. Because
-the variable result hasn't been given a type, it is assumed to hold a Python
-object.
-
-Lines 7-9 set up for a loop which will test candidate numbers for primeness
-until the required number of primes has been found. Lines 11-12, which try
-dividing a candidate by all the primes found so far, are of particular
-interest. Because no Python objects are referred to, the loop is translated
-entirely into C code, and thus runs very fast.
-
-When a prime is found, lines 14-15 add it to the p array for fast access by
-the testing loop, and line 16 adds it to the result list. Again, you'll notice
-that line 16 looks very much like a Python statement, and in fact it is, with
-the twist that the C parameter ``n`` is automatically converted to a Python
-object before being passed to the append method. Finally, at line 18, a normal
-Python return statement returns the result list.
-
-Compiling primes.pyx with the Cython compiler produces an extension module
-which we can try out in the interactive interpreter as follows::
-
-    >>> import primes
-    >>> primes.primes(10)
-    [2, 3, 5, 7, 11, 13, 17, 19, 23, 29]
-
-See, it works! And if you're curious about how much work Cython has saved you,
-take a look at the C code generated for this module. 
-
-Language Details
-================
-
-For more about the Cython language, see :ref:`language-basics-label`.
-
 Future Plans
 ============
 Cython is not finished. Substantial tasks remaining. See

docs/source_files_and_compilation.rst

-.. TODO: Rewrite this to be more comprehensive, with examples!
+.. _compilation_label:
 
 ****************************
 Source Files and Compilation
@@ -8,16 +8,6 @@ Cython source file names consist of the name of the module followed by a
 ``.pyx`` extension, for example a module called primes would have a source
 file named :file:`primes.pyx`.
 
-If your module is destined to live in a package, the source file name should
-include the full dotted name that the module will eventually have. For
-example, a module called primes that will be installed in a package called
-numbers should have a source file called numbers.primes.pyx. This will ensure
-that the :attr:`__name__` properties of the module and any classes defined in
-it are set correctly. If you don't do this, you may find that pickling doesn't
-work, among other problems. It also ensures that the Cython compiler has the
-right idea about the layout of the module namespace, which can be important
-when accessing extension types defined in other modules.
-
 Once you have written your ``.pyx`` file, there are a couple of ways of turning it
 into an extension module. One way is to compile it manually with the Cython
 compiler, e.g.::
@@ -26,12 +16,66 @@ compiler, e.g.::
 
 This will produce a file called :file:`primes.c`, which then needs to be
 compiled with the C compiler using whatever options are appropriate on your
-platform for generating an extension module. There's a Makefile in the Demos
-directory (called Makefile.nodistutils) that shows how to do this for Linux.
+platform for generating an extension module. For these options look at the
+official Python documentation.
 
 The other, and probably better, way is to use the :mod:`distutils` extension
-provided with Cython. See the :file:`setup.py` file in the Demos directory for an
-example of how to use it. This method has the advantage of being
-cross-platform -- the same setup file should work on any platform where
-:mod:`distutils` can compile an
-extension module.
+provided with Cython. The benifit of this method is that it will give the
+platform specific compilation options, acting like a stripped down autotools.
+
+Basic setup.py
+===============
+The distutils extension provided with Cython allows you to pass ``.pyx`` files
+directly to the ``Extension`` constructor in your setup file.
+
+If you have a single Cython file that you want to turn into a compiled
+extension, say with filename :file:`example.pyx` the associated :file:`setup.py`
+would be::
+
+    from distutils.core import setup
+    from distutils.extension import Extension
+    from Cython.Distutils import build_ext
+
+    setup(
+        cmdclass = {'build_ext': build_ext},
+        ext_modules = [Extension("example", ["example.pyx"])]
+    ) 
+
+To understand the :file:`setup.py` more fully look at the official
+``distutils`` documentation. To compile the extension for use in the
+current directory use::
+
+    $ python setup.py build_ext --inplace
+
+Cython Files Depending on C Files
+===================================
+
+TODO
+
+Multiple Cython Files in a Package
+===================================
+
+TODO
+
+Distributing Pyrex modules
+===========================
+It is strongly recommended that you distribute the generated ``.c`` files as well
+as your Cython sources, so that users can install your module without needing
+to have Cython available.
+
+It is also recommended that Cython compilation not be enabled by default in the
+version you distribute. Even if the user has Cython installed, he probably
+doesn't want to use it just to install your module. Also, the version he has
+may not be the same one you used, and may not compile your sources correctly.
+
+This simply means that the :file:`setup.py` file that you ship with will just
+be a normal distutils file on the generated `.c` files, for the basic example
+we would have instead::
+
+    from distutils.core import setup
+    from distutils.extension import Extension
+
+    setup(
+        ext_modules = [Extension("example", ["example.c"])]
+    ) 
+

docs/tutorial.rst

+.. _tutorial_label:
+
+*********
+Tutorial
+*********
+
+The Basics of Cython
+====================
+
+The fundamental nature of Cython can be summed up as follows: Cython is Python
+with C data types.
+
+Cython is Python: Almost any piece of Python code is also valid Cython code.
+(There are a few :ref:`cython-limitations-label`, but this approximation will
+serve for now.) The Cython compiler will convert it into C code which makes
+equivalent calls to the Python/C API. 
+
+But Cython is much more than that, because parameters and variables can be
+declared to have C data types. Code which manipulates Python values and C
+values can be freely intermixed, with conversions occurring automatically
+wherever possible. Reference count maintenance and error checking of Python
+operations is also automatic, and the full power of Python's exception
+handling facilities, including the try-except and try-finally statements, is
+available to you -- even in the midst of manipulating C data.
+
+
+
+Cython Hello World
+===================
+
+As Cython can accept almost any valid plain python file (though see the
+`exceptions`), one of the hardest things in getting started is just figuring
+out how to compile your file.
+
+So lets start with the canonical python hello world::
+
+    print "Hello World"
+
+So the first thing to do is rename the file to :file:`helloworld.pyx`. Now we
+need to make the :file:`setup.py`, which is like a python Makefile if you are
+familiar.::
+
+    from distutils.core import setup
+    from distutils.extension import Extension
+    from Cython.Distutils import build_ext
+
+    setup(
+        cmdclass = {'build_ext': build_ext},
+        ext_modules = [Extension("helloworld", ["helloworld.pyx"])]
+    ) 
+
+To use this to build your Cython file use the commandline options::
+
+    $ python setup.py build_ext --inplace
+
+Which will leave a file in your local directory called `helloworld.so`. Now to
+use this file start the python interpreter and::
+
+    >>> import helloworld
+    "Hello World"
+
+Congratulations! You know know how to build a Cython extension. But So Far
+this example doesn't really show us why we would even want to use Cython, so
+lets do a more realistic example.
+
+Fibonacci Fun
+==============
+
+From the official Python tutorial a simple fibonacci function is defined as::
+
+    def fib(n):
+        """Print the Fibonacci series up to n."""
+        a, b = 0, 1
+        while b < n:
+            print b,
+            a, b = b, a + b
+
+Now following the steps for the Hello World example we first rename the file
+to have a `.pyx` extension, lets say :file:`fib.pyx`, then we create the
+:file:`setup.py` file. Using the file created for the Hello World example, all
+that you need to change is the name of the Cython filename, and the resulting
+module name, doing this we have::
+
+    from distutils.core import setup
+    from distutils.extension import Extension
+    from Cython.Distutils import build_ext
+
+    setup(
+        cmdclass = {'build_ext': build_ext},
+        ext_modules = [Extension("fib", ["fib.pyx"])]
+    )
+
+Build the extension with the same command used for the helloworld.pyx::
+
+    $ python setup.py build_ext --inplace
+
+And use the new extension with::
+
+    >>> import fib
+    >>> fib.fib(2000)
+    1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597
+
+Primes
+=======
+
+Here's a small example showing some of what can be done. It's a routine for
+finding prime numbers. You tell it how many primes you want, and it returns
+them as a Python list.
+
+:file:`primes.pyx`: :: 
+
+    def primes(int kmax):
+        cdef int n, k, i
+        cdef int p[1000]
+        result = []
+        if kmax > 1000:
+            kmax = 1000
+        k = 0
+        n = 2
+        while k < kmax:
+            i = 0
+            while i < k and n % p[i] != 0:
+                i = i + 1
+            if i == k:
+               p[k] = n
+               k = k + 1
+               result.append(n)
+            n = n + 1
+        return result
+
+You'll see that it starts out just like a normal Python function definition,
+except that the parameter ``kmax`` is declared to be of type ``int`` . This
+means that the object passed will be converted to a C integer (or a
+``TypeError.`` will be raised if it can't be).
+
+Lines 2 and 3 use the ``cdef`` statement to define some local C variables.
+Line 4 creates a Python list which will be used to return the result. You'll
+notice that this is done exactly the same way it would be in Python. Because
+the variable result hasn't been given a type, it is assumed to hold a Python
+object.
+
+Lines 7-9 set up for a loop which will test candidate numbers for primeness
+until the required number of primes has been found. Lines 11-12, which try
+dividing a candidate by all the primes found so far, are of particular
+interest. Because no Python objects are referred to, the loop is translated
+entirely into C code, and thus runs very fast.
+
+When a prime is found, lines 14-15 add it to the p array for fast access by
+the testing loop, and line 16 adds it to the result list. Again, you'll notice
+that line 16 looks very much like a Python statement, and in fact it is, with
+the twist that the C parameter ``n`` is automatically converted to a Python
+object before being passed to the append method. Finally, at line 18, a normal
+Python return statement returns the result list.
+
+Compiling primes.pyx with the Cython compiler produces an extension module
+which we can try out in the interactive interpreter as follows::
+
+    >>> import primes
+    >>> primes.primes(10)
+    [2, 3, 5, 7, 11, 13, 17, 19, 23, 29]
+
+See, it works! And if you're curious about how much work Cython has saved you,
+take a look at the C code generated for this module. 
+
+Language Details
+================
+
+For more about the Cython language, see :ref:`language-basics-label`.
+

index.rst

@@ -11,6 +11,7 @@ Contents:
    :maxdepth: 2
 
    docs/overview
+   docs/tutorial
    docs/language_basics
    docs/extension_types
    docs/sharing_declarations