Merge pull request #2296 from gabrieldemarmiesse/extend_example_tests

Enable tests for all the files in the example directory.

docs/examples/README.rst

+This example directory is organized like the ``Cython/docs/src/`` directory,
+with one directory per ``.rst`` file. All files in this directory are tested
+in the :file:`runtests.py` with the mode `compile`.

docs/examples/quickstart/build/hello.pyx

+def say_hello_to(name):
+    print("Hello %s!" % name)

docs/examples/tutorial/primes/setup.py → docs/examples/quickstart/build/setup.py

 from distutils.core import setup
 from Cython.Build import cythonize
 
-setup(
-    ext_modules=cythonize("primes.pyx"),
-)
+setup(name='Hello world app',
+      ext_modules=cythonize("hello.pyx"))

docs/examples/tutorial/primes/primes_cpp.pyx → docs/examples/tutorial/cython_tutorial/primes_cpp.pyx

@@ -2,7 +2,7 @@
 
 from libcpp.vector cimport vector
 
-def primes(int nb_primes):
+def primes(unsigned int nb_primes):
     cdef int n, i
     cdef vector[int] p
     p.reserve(nb_primes)  # allocate memory for 'nb_primes' elements.

docs/src/quickstart/build.rst

@@ -37,20 +37,13 @@ The other methods are described in more detail in the :ref:`compilation` section
 Building a Cython module using distutils
 ----------------------------------------
 
-Imagine a simple "hello world" script in a file ``hello.pyx``::
+Imagine a simple "hello world" script in a file ``hello.pyx``:
 
-  def say_hello_to(name):
-      print("Hello %s!" % name)
+.. literalinclude:: ../../examples/quickstart/build/hello.pyx
 
-The following could be a corresponding ``setup.py`` script::
+The following could be a corresponding ``setup.py`` script:
 
-  from distutils.core import setup
-  from Cython.Build import cythonize
-
-  setup(
-    name = 'Hello world app',
-    ext_modules = cythonize("hello.pyx"),
-  )
+.. literalinclude:: ../../examples/quickstart/build/setup.py
 
 To build, run ``python setup.py build_ext --inplace``.  Then simply
 start a Python session and do ``from hello import say_hello_to`` and

docs/src/tutorial/cython_tutorial.rst

@@ -97,7 +97,7 @@ Fibonacci Fun
 
 From the official Python tutorial a simple fibonacci function is defined as:
 
-.. literalinclude:: ../../examples/tutorial/fib1/fib.pyx
+.. literalinclude:: ../../examples/tutorial/cython_tutorial/fib.pyx
 
 Now following the steps for the Hello World example we first rename the file
 to have a `.pyx` extension, lets say :file:`fib.pyx`, then we create the
@@ -105,7 +105,7 @@ to have a `.pyx` extension, lets say :file:`fib.pyx`, then we create the
 that you need to change is the name of the Cython filename, and the resulting
 module name, doing this we have:
 
-.. literalinclude:: ../../examples/tutorial/fib1/setup.py
+.. literalinclude:: ../../examples/tutorial/cython_tutorial/setup.py
 
 Build the extension with the same command used for the helloworld.pyx:
 
@@ -130,7 +130,7 @@ them as a Python list.
 
 :file:`primes.pyx`:
 
-.. literalinclude:: ../../examples/tutorial/primes/primes.pyx
+.. literalinclude:: ../../examples/tutorial/cython_tutorial/primes.pyx
     :linenos:
 
 You'll see that it starts out just like a normal Python function definition,
@@ -352,7 +352,7 @@ standard library.
     how many elements you are going to put in the vector. For more details
     see `this page from cppreference <http://en.cppreference.com/w/cpp/container/vector>`_.
 
-.. literalinclude:: ../../examples/tutorial/primes/primes_cpp.pyx
+.. literalinclude:: ../../examples/tutorial/cython_tutorial/primes_cpp.pyx
     :linenos:
 
 The first line is a compiler directive. It tells Cython to compile your code to C++.

docs/src/userguide/memoryviews.rst

@@ -703,19 +703,19 @@ manage an array (allocate and deallocate) with NumPy (it can also be Python arra
 anything that supports the buffer interface), but you want to perform computation on this
 array with an external C function implemented in :file:`C_func_file.c`:
 
-.. literalinclude:: ../../examples/memoryviews/C_func_file.c
+.. literalinclude:: ../../examples/userguide/memoryviews/C_func_file.c
     :linenos:
 
 This file comes with a header file called :file:`C_func_file.h` containing:
 
-.. literalinclude:: ../../examples/memoryviews/C_func_file.h
+.. literalinclude:: ../../examples/userguide/memoryviews/C_func_file.h
     :linenos:
 
 where ``arr`` points to the array and ``n`` is its size.
 
 You can call the function in a Cython file in the following way:
 
-.. literalinclude:: ../../examples/memoryviews/memview_to_c.pyx
+.. literalinclude:: ../../examples/userguide/memoryviews/memview_to_c.pyx
     :linenos:
 
 Several things to note:

docs/src/userguide/numpy_tutorial.rst

@@ -144,7 +144,7 @@ valid Python and valid Cython code. I'll refer to it as both
 :file:`convolve_py.py` for the Python version and :file:`convolve_cy.pyx` for the
 Cython version -- Cython uses ".pyx" as its file suffix.
 
-.. literalinclude:: ../../examples/memoryviews/convolve_py.py
+.. literalinclude:: ../../examples/userguide/numpy_tutorial/convolve_py.py
     :linenos:
 
 This should be compiled to produce :file:`convolve_cy.so` (for Linux systems). We
@@ -194,7 +194,7 @@ Adding types
 To add types we use custom Cython syntax, so we are now breaking Python source
 compatibility. Here's :file:`convolve_typed.pyx`. *Read the comments!*
 
-.. literalinclude:: ../../examples/memoryviews/convolve_typed.pyx
+.. literalinclude:: ../../examples/userguide/numpy_tutorial/convolve_typed.pyx
     :linenos:
 
 .. figure:: convolve_types_html.png
@@ -269,7 +269,7 @@ Here is how to use them in our code:
 
 :file:`convolve_memview.pyx`
 
-.. literalinclude:: ../../examples/memoryviews/convolve_memview.pyx
+.. literalinclude:: ../../examples/userguide/numpy_tutorial/convolve_memview.pyx
     :linenos:
 
 Let's see how much faster accessing is now.
@@ -381,7 +381,7 @@ our code. This is why, we must still declare manually the type of the
 And actually, manually giving the type of the ``value`` variable will
 be useful when using fused types.
 
-.. literalinclude:: ../../examples/memoryviews/convolve_infer_types.pyx
+.. literalinclude:: ../../examples/userguide/numpy_tutorial/convolve_infer_types.pyx
     :linenos:
 
 We now do a speed test:
@@ -416,7 +416,7 @@ know what NumPy data type we should use for our output array.
 
 In this case, our function now works for ints, doubles and floats.
 
-.. literalinclude:: ../../examples/memoryviews/convolve_fused_types.pyx
+.. literalinclude:: ../../examples/userguide/numpy_tutorial/convolve_fused_types.pyx
     :linenos:
 
 We can check that the output type is the right one::

runtests.py

@@ -16,6 +16,7 @@ import tempfile
 import traceback
 import warnings
 import zlib
+import glob
 
 try:
     import platform
@@ -643,14 +644,14 @@ class TestBuilder(object):
         expect_warnings = 'warnings' in tags['tag']
 
         if expect_errors:
-            if 'cpp' in tags['tag'] and 'cpp' in self.languages:
+            if skip_c(tags) and 'cpp' in self.languages:
                 languages = ['cpp']
             else:
                 languages = self.languages[:1]
         else:
             languages = self.languages
 
-        if 'cpp' in tags['tag'] and 'c' in languages:
+        if skip_c(tags) and 'c' in languages:
             languages = list(languages)
             languages.remove('c')
         elif 'no-cpp' in tags['tag'] and 'cpp' in self.languages:
@@ -697,6 +698,22 @@ class TestBuilder(object):
                           pythran_dir=pythran_dir)
 
 
+def skip_c(tags):
+    if 'cpp' in tags['tag']:
+        return True
+
+    # We don't want to create a distutils key in the
+    # dictionary so we check before looping.
+    if 'distutils' in tags:
+        for option in tags['distutils']:
+            splitted = option.split('=')
+            if len(splitted) == 2:
+                argument, value = splitted
+                if argument.strip() == 'language' and value.strip() == 'c++':
+                    return True
+    return False
+
+
 class CythonCompileTestCase(unittest.TestCase):
     def __init__(self, test_directory, workdir, module, tags, language='c', preparse='id',
                  expect_errors=False, expect_warnings=False, annotate=False, cleanup_workdir=True,
@@ -2142,7 +2159,8 @@ def runtests(options, cmd_args, coverage=None):
                                 options.pythran_dir, add_embedded_test=True)
         test_suite.addTest(filetests.build_suite())
     if options.examples and languages:
-        filetests = TestBuilder(options.examples_dir, WORKDIR, selectors, exclude_selectors,
+        for subdirectory in glob.glob(os.path.join(options.examples_dir, "*/")):
+            filetests = TestBuilder(subdirectory, WORKDIR, selectors, exclude_selectors,
                                     options, options.pyregr, languages, test_bugs,
                                     options.language_level, common_utility_dir,
                                     options.pythran_dir,