Merge from 3.2

Doc/faq/design.rst

@@ -380,11 +380,24 @@ is exactly the same type of object that a lambda form yields) is assigned!
 Can Python be compiled to machine code, C or some other language?
 -----------------------------------------------------------------
 
-Not easily.  Python's high level data types, dynamic typing of objects and
+Practical answer:
+
+`Cython <http://cython.org/>`_ and `Pyrex <http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/>`_
+compile a modified version of Python with optional annotations into C
+extensions.  `Weave <http://www.scipy.org/Weave>`_ makes it easy to
+intermingle Python and C code in various ways to increase performance.
+`Nuitka <http://www.nuitka.net/>`_ is an up-and-coming compiler of Python
+into C++ code, aiming to support the full Python language.
+
+Theoretical answer:
+
+   .. XXX not sure what to make of this
+
+Not trivially.  Python's high level data types, dynamic typing of objects and
 run-time invocation of the interpreter (using :func:`eval` or :func:`exec`)
-together mean that a "compiled" Python program would probably consist mostly of
-calls into the Python run-time system, even for seemingly simple operations like
-``x+1``.
+together mean that a naïvely "compiled" Python program would probably consist
+mostly of calls into the Python run-time system, even for seemingly simple
+operations like ``x+1``.
 
 Several projects described in the Python newsgroup or at past `Python
 conferences <http://python.org/community/workshops/>`_ have shown that this
@@ -395,99 +408,64 @@ speedups of 1000x are feasible for small demo programs.  See the proceedings
 from the `1997 Python conference
 <http://python.org/workshops/1997-10/proceedings/>`_ for more information.)
 
-Internally, Python source code is always translated into a bytecode
-representation, and this bytecode is then executed by the Python virtual
-machine.  In order to avoid the overhead of repeatedly parsing and translating
-modules that rarely change, this byte code is written into a file whose name
-ends in ".pyc" whenever a module is parsed.  When the corresponding .py file is
-changed, it is parsed and translated again and the .pyc file is rewritten.
-
-There is no performance difference once the .pyc file has been loaded, as the
-bytecode read from the .pyc file is exactly the same as the bytecode created by
-direct translation.  The only difference is that loading code from a .pyc file
-is faster than parsing and translating a .py file, so the presence of
-precompiled .pyc files improves the start-up time of Python scripts.  If
-desired, the Lib/compileall.py module can be used to create valid .pyc files for
-a given set of modules.
-
-Note that the main script executed by Python, even if its filename ends in .py,
-is not compiled to a .pyc file.  It is compiled to bytecode, but the bytecode is
-not saved to a file.  Usually main scripts are quite short, so this doesn't cost
-much speed.
-
-.. XXX check which of these projects are still alive
-
-There are also several programs which make it easier to intermingle Python and C
-code in various ways to increase performance.  See, for example, `Cython
-<http://cython.org/>`_, `Pyrex
-<http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/>`_ and `Weave
-<http://www.scipy.org/Weave>`_.
-
 
 How does Python manage memory?
 ------------------------------
 
 The details of Python memory management depend on the implementation.  The
-standard C implementation of Python uses reference counting to detect
-inaccessible objects, and another mechanism to collect reference cycles,
+standard implementation of Python, :term:`CPython`, uses reference counting to
+detect inaccessible objects, and another mechanism to collect reference cycles,
 periodically executing a cycle detection algorithm which looks for inaccessible
 cycles and deletes the objects involved. The :mod:`gc` module provides functions
 to perform a garbage collection, obtain debugging statistics, and tune the
 collector's parameters.
 
-Jython relies on the Java runtime so the JVM's garbage collector is used.  This
-difference can cause some subtle porting problems if your Python code depends on
-the behavior of the reference counting implementation.
+Other implementations (such as `Jython <http://www.jython.org>`_ or
+`PyPy <http://www.pypy.org>`_), however, can rely on a different mechanism
+such as a full-blown garbage collector.  This difference can cause some
+subtle porting problems if your Python code depends on the behavior of the
+reference counting implementation.
 
-.. XXX relevant for Python 3?
+In some Python implementations, the following code (which is fine in CPython)
+will probably run out of file descriptors::
 
-   Sometimes objects get stuck in traceback temporarily and hence are not
-   deallocated when you might expect.  Clear the traceback with::
+   for file in very_long_list_of_files:
+       f = open(file)
+       c = f.read(1)
 
-     import sys
-     sys.last_traceback = None
+Indeed, using CPython's reference counting and destructor scheme, each new
+assignment to *f* closes the previous file.  With a traditional GC, however,
+those file objects will only get collected (and closed) at varying and possibly
+long intervals.
 
-   Tracebacks are used for reporting errors, implementing debuggers and related
-   things.  They contain a portion of the program state extracted during the
-   handling of an exception (usually the most recent exception).
+If you want to write code that will work with any Python implementation,
+you should explicitly close the file or use the :keyword:`with` statement;
+this will work regardless of memory management scheme::
 
-In the absence of circularities, Python programs do not need to manage memory
-explicitly.
+   for file in very_long_list_of_files:
+       with open(file) as f:
+           c = f.read(1)
 
-Why doesn't Python use a more traditional garbage collection scheme?  For one
-thing, this is not a C standard feature and hence it's not portable.  (Yes, we
-know about the Boehm GC library.  It has bits of assembler code for *most*
-common platforms, not for all of them, and although it is mostly transparent, it
-isn't completely transparent; patches are required to get Python to work with
-it.)
+
+Why doesn't CPython use a more traditional garbage collection scheme?
+---------------------------------------------------------------------
+
+For one thing, this is not a C standard feature and hence it's not portable.
+(Yes, we know about the Boehm GC library.  It has bits of assembler code for
+*most* common platforms, not for all of them, and although it is mostly
+transparent, it isn't completely transparent; patches are required to get
+Python to work with it.)
 
 Traditional GC also becomes a problem when Python is embedded into other
 applications.  While in a standalone Python it's fine to replace the standard
 malloc() and free() with versions provided by the GC library, an application
 embedding Python may want to have its *own* substitute for malloc() and free(),
-and may not want Python's.  Right now, Python works with anything that
+and may not want Python's.  Right now, CPython works with anything that
 implements malloc() and free() properly.
 
-In Jython, the following code (which is fine in CPython) will probably run out
-of file descriptors long before it runs out of memory::
-
-   for file in very_long_list_of_files:
-       f = open(file)
-       c = f.read(1)
-
-Using the current reference counting and destructor scheme, each new assignment
-to f closes the previous file.  Using GC, this is not guaranteed.  If you want
-to write code that will work with any Python implementation, you should
-explicitly close the file or use the :keyword:`with` statement; this will work
-regardless of GC::
 
-   for file in very_long_list_of_files:
-       with open(file) as f:
-           c = f.read(1)
-
-
-Why isn't all memory freed when Python exits?
----------------------------------------------
+Why isn't all memory freed when CPython exits?
+----------------------------------------------
 
 Objects referenced from the global namespaces of Python modules are not always
 deallocated when Python exits.  This may happen if there are circular
@@ -647,10 +625,10 @@ order to remind you of that fact, it does not return the sorted list.  This way,
 you won't be fooled into accidentally overwriting a list when you need a sorted
 copy but also need to keep the unsorted version around.
 
-In Python 2.4 a new built-in function -- :func:`sorted` -- has been added.
-This function creates a new list from a provided iterable, sorts it and returns
-it.  For example, here's how to iterate over the keys of a dictionary in sorted
-order::
+If you want to return a new list, use the built-in :func:`sorted` function
+instead.  This function creates a new list from a provided iterable, sorts
+it and returns it.  For example, here's how to iterate over the keys of a
+dictionary in sorted order::
 
    for key in sorted(mydict):
        ... # do whatever with mydict[key]...