Update pyximport documentation following the discussion in #1827.

docs/src/quickstart/build.rst

@@ -11,7 +11,7 @@ Cython code must, unlike Python, be compiled. This happens in two stages:
 
 There are several ways to build Cython code:
 
- - Write a distutils ``setup.py``.
+ - Write a distutils ``setup.py``. This is the normal and recommended way.
  - Use ``pyximport``, importing Cython ``.pyx`` files as if they
    were ``.py`` files (using distutils to compile and build in the background).
  - Run the ``cython`` command-line utility manually to produce the ``.c`` file

docs/src/reference/compilation.rst

@@ -308,10 +308,11 @@ e.g.::
 These ``.pxd`` files need not have corresponding ``.pyx``
 modules if they contain purely declarations of external libraries.
 
-Compiling with ``pyximport``
-=============================
+Compiling with :mod:`pyximport`
+===============================
 
-For generating Cython code right in your pure python module just type::
+For building Cython modules during development without explicitly
+running ``setup.py`` after each change, you can use :mod:`pyximport`::
 
     >>> import pyximport; pyximport.install()
     >>> import helloworld
@@ -322,15 +323,20 @@ Python is trying to import.  You should use this for simple Cython
 builds only where no extra C libraries and no special building setup
 is needed.
 
-In the case that Cython fails to compile a Python module, *pyximport*
-will fall back to loading the source modules instead.
-
 It is also possible to compile new ``.py`` modules that are being
 imported (including the standard library and installed packages).  For
-using this feature, just tell that to ``pyximport``::
+using this feature, just tell that to :mod:`pyximport`::
 
     >>> pyximport.install(pyimport = True)
 
+In the case that Cython fails to compile a Python module, :mod:`pyximport`
+will fall back to loading the source modules instead.
+
+Note that it is not recommended to let :mod:`pyximport` build code
+on end user side as it hooks into their import system.  The best way
+to cater for end users is to provide pre-built binary packages in the
+`wheel <https://wheel.readthedocs.io/>`_ packaging format.
+
 Compiling with ``cython.inline``
 =================================
 

docs/src/tutorial/cython_tutorial.rst

@@ -64,14 +64,14 @@ Congratulations! You now know how to build a Cython extension. But so far
 this example doesn't really give a feeling why one would ever want to use Cython, so
 lets create a more realistic example.
 
-:mod:`pyximport`: Cython Compilation the Easy Way
-==================================================
+:mod:`pyximport`: Cython Compilation for Developers
+---------------------------------------------------
 
 If your module doesn't require any extra C libraries or a special
-build setup, then you can use the pyximport module by Paul Prescod and
-Stefan Behnel to load .pyx files directly on import, without having to
-write a :file:`setup.py` file.  It is shipped and installed with
-Cython and can be used like this::
+build setup, then you can use the pyximport module, originally developed
+by Paul Prescod, to load .pyx files directly on import, without having
+to run your :file:`setup.py` file each time you change your code.
+It is shipped and installed with Cython and can be used like this::
 
     >>> import pyximport; pyximport.install()
     >>> import helloworld
@@ -87,6 +87,11 @@ modules instead.  The .py import mechanism is installed like this::
 
     >>> pyximport.install(pyimport = True)
 
+Note that it is not recommended to let :mod:`pyximport` build code
+on end user side as it hooks into their import system.  The best way
+to cater for end users is to provide pre-built binary packages in the
+`wheel <https://wheel.readthedocs.io/>`_ packaging format.
+
 Fibonacci Fun
 ==============
 

docs/src/userguide/source_files_and_compilation.rst

@@ -80,17 +80,11 @@ them through :func:`cythonize`::
 Pyximport
 ===========
 
-.. TODO add some text about how this is Paul Prescods code. Also change the
-   tone to be more universal (i.e. remove all the I statements)
-
-Cython is a compiler. Therefore it is natural that people tend to go
-through an edit/compile/test cycle with Cython modules. But my personal
-opinion is that one of the deep insights in Python's implementation is
-that a language can be compiled (Python modules are compiled to ``.pyc``
-files) and hide that compilation process from the end-user so that they
-do not have to worry about it. Pyximport does this for Cython modules.
-For instance if you write a Cython module called :file:`foo.pyx`, with
-Pyximport you can import it in a regular Python module like this::
+Cython is a compiler.  Therefore it is natural that people tend to go
+through an edit/compile/test cycle with Cython modules.  :mod:`pyximport`
+simplifies this process by executing the "compile" step at need during
+import.  For instance, if you write a Cython module called :file:`foo.pyx`,
+with Pyximport you can import it in a regular Python module like this::
 
 
     import pyximport; pyximport.install()
@@ -99,64 +93,72 @@ Pyximport you can import it in a regular Python module like this::
 Doing so will result in the compilation of :file:`foo.pyx` (with appropriate
 exceptions if it has an error in it).
 
-If you would always like to import Cython files without building them
-specially, you can also the first line above to your :file:`sitecustomize.py`.
-That will install the hook every time you run Python. Then you can use
-Cython modules just with simple import statements. I like to test my
-Cython modules like this:
+If you would always like to import Cython files without building them specially,
+you can also add the first line above to your :file:`sitecustomize.py`.
+That will install the hook every time you run Python.  Then you can use
+Cython modules just with simple import statements, even like this:
 
 .. sourcecode:: text
 
     $ python -c "import foo"
 
+Note that it is not recommended to let :mod:`pyximport` build code
+on end user side as it hooks into their import system.  The best way
+to cater for end users is to provide pre-built binary packages in the
+`wheel <https://wheel.readthedocs.io/>`_ packaging format.
+
 Dependency Handling
 --------------------
 
-In Pyximport 1.1 it is possible to declare that your module depends on
-multiple files, (likely ``.h`` and ``.pxd`` files). If your Cython module is
-named ``foo`` and thus has the filename :file:`foo.pyx` then you should make
-another file in the same directory called :file:`foo.pyxdep`. The
-:file:`modname.pyxdep` file can be a list of filenames or "globs" (like
-``*.pxd`` or ``include/*.h``). Each filename or glob must be on a separate
-line. Pyximport will check the file date for each of those files before
-deciding whether to rebuild the module. In order to keep track of the
-fact that the dependency has been handled, Pyximport updates the
-modification time of your ".pyx" source file. Future versions may do
-something more sophisticated like informing distutils of the
-dependencies directly.
+Since :mod:`pyximport` does not use `cythonize()` internally, it currently
+requires a different setup for dependencies.  It is possible to declare that
+your module depends on multiple files, (likely ``.h`` and ``.pxd`` files).
+If your Cython module is named ``foo`` and thus has the filename
+:file:`foo.pyx` then you should create another file in the same directory
+called :file:`foo.pyxdep`.  The :file:`modname.pyxdep` file can be a list of
+filenames or "globs" (like ``*.pxd`` or ``include/*.h``).  Each filename or
+glob must be on a separate line.  Pyximport will check the file date for each
+of those files before deciding whether to rebuild the module.  In order to
+keep track of the fact that the dependency has been handled, Pyximport updates
+the modification time of your ".pyx" source file.  Future versions may do
+something more sophisticated like informing distutils of the dependencies
+directly.
 
 Limitations
 ------------
 
 Pyximport does not give you any control over how your Cython file is
-compiled. Usually the defaults are fine. You might run into problems if
+compiled.  Usually the defaults are fine.  You might run into problems if
 you wanted to write your program in half-C, half-Cython and build them
-into a single library. Pyximport 1.2 will probably do this.
+into a single library.
 
 Pyximport does not hide the Distutils/GCC warnings and errors generated
-by the import process. Arguably this will give you better feedback if
-something went wrong and why. And if nothing went wrong it will give you
-the warm fuzzy that pyximport really did rebuild your module as it was
-supposed to.
+by the import process.  Arguably this will give you better feedback if
+something went wrong and why.  And if nothing went wrong it will give you
+the warm fuzzy feeling that pyximport really did rebuild your module as it
+was supposed to.
+
+Basic module reloading support is available with the option ``reload_support=True``.
+Note that this will generate a new module filename for each build and thus
+end up loading multiple shared libraries into memory over time.  CPython does
+not support reloading shared libraries as such.
+
+Pyximport puts your ``.c`` file beside your ``.pyx`` file (analogous to
+``.pyc`` beside ``.py``), but, by default, puts the platform-specific
+binary in a build directory as per normal for Distutils.  To copy it back
+into the package hierarchy (usually next to the source file) for manual
+reuse, you can pass the option ``inplace=True``.
 
 For further thought and discussion
 ------------------------------------
 
-I don't think that Python's :func:`reload` will do anything for changed
-``.so``'s on some (all?) platforms. It would require some (easy)
-experimentation that I haven't gotten around to. But reload is rarely used in
-applications outside of the Python interactive interpreter and certainly not
-used much for C extension modules. Info about Windows
-`<http://mail.python.org/pipermail/python-list/2001-July/053798.html>`_
-
-``setup.py install`` does not modify :file:`sitecustomize.py` for you. Should it?
-Modifying Python's "standard interpreter" behaviour may be more than
-most people expect of a package they install..
+``setup.py install`` does not modify :file:`sitecustomize.py` for you and
+probably should never do that.  Modifying Python's "standard interpreter"
+behaviour may be more than most people expect of a package they install.
 
 Pyximport puts your ``.c`` file beside your ``.pyx`` file (analogous to
-``.pyc`` beside ``.py``). But it puts the platform-specific binary in a
-build directory as per normal for Distutils. If I could wave a magic
+``.pyc`` beside ``.py``).  But it puts the platform-specific binary in a
+build directory as per normal for Distutils.  If I could wave a magic
 wand and get Cython or distutils or whoever to put the build directory I
 might do it but not necessarily: having it at the top level is *VERY*
 *HELPFUL* for debugging Cython problems.
-