merge 3.2

.hgeol

@@ -28,8 +28,9 @@
 
 Lib/email/test/data/msg_26.txt = BIN
 Lib/test/cjkencodings/* = BIN
-Lib/test/sndhdrdata/sndhdr.* = BIN
 Lib/test/decimaltestdata/*.decTest = BIN
+Lib/test/sndhdrdata/sndhdr.* = BIN
+Lib/test/test_email/data/msg_26.txt = BIN
 
 # All other files (which presumably are human-editable) are "native".
 # This must be the last rule!

.hgignore

@@ -5,7 +5,8 @@ Makefile$
 Makefile.pre$
 TAGS$
 autom4te.cache$
-build/
+^build/
+^Doc/build/
 buildno$
 config.cache
 config.log
@@ -32,6 +33,7 @@ Modules/config.c
 Modules/ld_so_aix$
 Parser/pgen$
 Parser/pgen.stamp$
+PCbuild/amd64/
 ^core
 ^python-gdb.py
 ^python.exe-gdb.py
@@ -62,7 +64,10 @@ PCbuild/*.exp
 PCbuild/*.o
 PCbuild/*.ncb
 PCbuild/*.bsc
+PCbuild/*.user
+PCbuild/*.suo
 PCbuild/Win32-temp-*
+PCbuild/x64-temp-*
 __pycache__
 Modules/_testembed
 .coverage

Doc/ACKS.txt

@@ -202,6 +202,7 @@ docs@python.org), and we'll be glad to correct the problem.
    * Jim Tittsler
    * David Turner
    * Ville Vainio
+   * Nadeem Vawda
    * Martijn Vries
    * Charles G. Waldman
    * Greg Ward

Doc/bugs.rst

@@ -57,12 +57,14 @@ were using (including version information as appropriate).
 
 Each bug report will be assigned to a developer who will determine what needs to
 be done to correct the problem.  You will receive an update each time action is
-taken on the bug.  See http://www.python.org/dev/workflow/ for a detailed
-description of the issue workflow.
+taken on the bug.
 
 
 .. seealso::
 
+   `Python Developer's Guide <http://docs.python.org/devguide/>`_
+      Detailed description of the issue workflow and developers tools.
+
    `How to Report Bugs Effectively <http://www.chiark.greenend.org.uk/~sgtatham/bugs.html>`_
       Article which goes into some detail about how to create a useful bug report.
       This describes what kind of information is useful and why it is useful.

Doc/c-api/import.rst

@@ -57,7 +57,7 @@ Importing Modules
    :c:func:`PyImport_ImportModule`.
 
 
-.. c:function:: PyObject* PyImport_ImportModuleLevel(char *name, PyObject *globals, PyObject *locals, PyObject *fromlist, int level)
+.. c:function:: PyObject* PyImport_ImportModuleLevelObject(PyObject *name, PyObject *globals, PyObject *locals, PyObject *fromlist, int level)
 
    Import a module.  This is best described by referring to the built-in Python
    function :func:`__import__`, as the standard :func:`__import__` function calls
@@ -68,6 +68,13 @@ Importing Modules
    the return value when a submodule of a package was requested is normally the
    top-level package, unless a non-empty *fromlist* was given.
 
+   .. versionadded:: 3.3
+
+
+.. c:function:: PyObject* PyImport_ImportModuleLevel(char *name, PyObject *globals, PyObject *locals, PyObject *fromlist, int level)
+
+   Similar to :c:func:`PyImport_ImportModuleLevelObject`, but the name is an
+   UTF-8 encoded string instead of a Unicode object.
 
 .. c:function:: PyObject* PyImport_Import(PyObject *name)
 
@@ -86,7 +93,7 @@ Importing Modules
    an exception set on failure (the module still exists in this case).
 
 
-.. c:function:: PyObject* PyImport_AddModule(const char *name)
+.. c:function:: PyObject* PyImport_AddModuleObject(PyObject *name)
 
    Return the module object corresponding to a module name.  The *name* argument
    may be of the form ``package.module``. First check the modules dictionary if
@@ -100,6 +107,14 @@ Importing Modules
       or one of its variants to import a module.  Package structures implied by a
       dotted name for *name* are not created if not already present.
 
+   .. versionadded:: 3.3
+
+
+.. c:function:: PyObject* PyImport_AddModule(const char *name)
+
+   Similar to :c:func:`PyImport_AddModuleObject`, but the name is a UTF-8
+   encoded string instead of a Unicode object.
+
 
 .. c:function:: PyObject* PyImport_ExecCodeModule(char *name, PyObject *co)
 
@@ -136,14 +151,23 @@ Importing Modules
    See also :c:func:`PyImport_ExecCodeModuleWithPathnames`.
 
 
-.. c:function:: PyObject* PyImport_ExecCodeModuleWithPathnames(char *name, PyObject *co, char *pathname, char *cpathname)
+.. c:function:: PyObject* PyImport_ExecCodeModuleObject(PyObject *name, PyObject *co, PyObject *pathname, PyObject *cpathname)
 
    Like :c:func:`PyImport_ExecCodeModuleEx`, but the :attr:`__cached__`
    attribute of the module object is set to *cpathname* if it is
    non-``NULL``.  Of the three functions, this is the preferred one to use.
 
+   .. versionadded:: 3.3
+
+
+.. c:function:: PyObject* PyImport_ExecCodeModuleWithPathnames(char *name, PyObject *co, char *pathname, char *cpathname)
+
+   Like :c:func:`PyImport_ExecCodeModuleObject`, but *name*, *pathname* and
+   *cpathname* are UTF-8 encoded strings.
+
    .. versionadded:: 3.2
 
+
 .. c:function:: long PyImport_GetMagicNumber()
 
    Return the magic number for Python bytecode files (a.k.a. :file:`.pyc` and
@@ -200,7 +224,7 @@ Importing Modules
    For internal use only.
 
 
-.. c:function:: int PyImport_ImportFrozenModule(char *name)
+.. c:function:: int PyImport_ImportFrozenModuleObject(PyObject *name)
 
    Load a frozen module named *name*.  Return ``1`` for success, ``0`` if the
    module is not found, and ``-1`` with an exception set if the initialization
@@ -208,6 +232,14 @@ Importing Modules
    :c:func:`PyImport_ImportModule`.  (Note the misnomer --- this function would
    reload the module if it was already imported.)
 
+   .. versionadded:: 3.3
+
+
+.. c:function:: int PyImport_ImportFrozenModule(char *name)
+
+   Similar to :c:func:`PyImport_ImportFrozenModuleObject`, but the name is a
+   UTF-8 encoded string instead of a Unicode object.
+
 
 .. c:type:: struct _frozen
 
@@ -247,13 +279,13 @@ Importing Modules
 
    Structure describing a single entry in the list of built-in modules.  Each of
    these structures gives the name and initialization function for a module built
-   into the interpreter.  Programs which embed Python may use an array of these
-   structures in conjunction with :c:func:`PyImport_ExtendInittab` to provide
-   additional built-in modules.  The structure is defined in
-   :file:`Include/import.h` as::
+   into the interpreter.  The name is an ASCII encoded string.  Programs which
+   embed Python may use an array of these structures in conjunction with
+   :c:func:`PyImport_ExtendInittab` to provide additional built-in modules.
+   The structure is defined in :file:`Include/import.h` as::
 
       struct _inittab {
-          char *name;
+          char *name;                 /* ASCII encoded string */
           PyObject* (*initfunc)(void);
       };
 

Doc/c-api/module.rst

@@ -29,7 +29,7 @@ There are only a few functions special to module objects.
    :c:data:`PyModule_Type`.
 
 
-.. c:function:: PyObject* PyModule_New(const char *name)
+.. c:function:: PyObject* PyModule_NewObject(PyObject *name)
 
    .. index::
       single: __name__ (module attribute)
@@ -40,6 +40,14 @@ There are only a few functions special to module objects.
    Only the module's :attr:`__doc__` and :attr:`__name__` attributes are filled in;
    the caller is responsible for providing a :attr:`__file__` attribute.
 
+   .. versionadded:: 3.3
+
+
+.. c:function:: PyObject* PyModule_New(const char *name)
+
+   Similar to :c:func:`PyImport_NewObject`, but the name is an UTF-8 encoded
+   string instead of a Unicode object.
+
 
 .. c:function:: PyObject* PyModule_GetDict(PyObject *module)
 
@@ -52,7 +60,7 @@ There are only a few functions special to module objects.
    manipulate a module's :attr:`__dict__`.
 
 
-.. c:function:: char* PyModule_GetName(PyObject *module)
+.. c:function:: PyObject* PyModule_GetNameObject(PyObject *module)
 
    .. index::
       single: __name__ (module attribute)
@@ -61,15 +69,13 @@ There are only a few functions special to module objects.
    Return *module*'s :attr:`__name__` value.  If the module does not provide one,
    or if it is not a string, :exc:`SystemError` is raised and *NULL* is returned.
 
+   .. versionadded:: 3.3
 
-.. c:function:: char* PyModule_GetFilename(PyObject *module)
 
-   Similar to :c:func:`PyModule_GetFilenameObject` but return the filename
-   encoded to 'utf-8'.
+.. c:function:: char* PyModule_GetName(PyObject *module)
 
-   .. deprecated:: 3.2
-      :c:func:`PyModule_GetFilename` raises :c:type:`UnicodeEncodeError` on
-      unencodable filenames, use :c:func:`PyModule_GetFilenameObject` instead.
+   Similar to :c:func:`PyModule_GetNameObject` but return the name encoded to
+   ``'utf-8'``.
 
 
 .. c:function:: PyObject* PyModule_GetFilenameObject(PyObject *module)
@@ -86,6 +92,16 @@ There are only a few functions special to module objects.
    .. versionadded:: 3.2
 
 
+.. c:function:: char* PyModule_GetFilename(PyObject *module)
+
+   Similar to :c:func:`PyModule_GetFilenameObject` but return the filename
+   encoded to 'utf-8'.
+
+   .. deprecated:: 3.2
+      :c:func:`PyModule_GetFilename` raises :c:type:`UnicodeEncodeError` on
+      unencodable filenames, use :c:func:`PyModule_GetFilenameObject` instead.
+
+
 .. c:function:: void* PyModule_GetState(PyObject *module)
 
    Return the "state" of the module, that is, a pointer to the block of memory

Doc/c-api/unicode.rst

@@ -260,18 +260,27 @@ APIs:
    | :attr:`%ld`       | long                | Exactly equivalent to          |
    |                   |                     | ``printf("%ld")``.             |
    +-------------------+---------------------+--------------------------------+
+   | :attr:`%li`       | long                | Exactly equivalent to          |
+   |                   |                     | ``printf("%li")``.             |
+   +-------------------+---------------------+--------------------------------+
    | :attr:`%lu`       | unsigned long       | Exactly equivalent to          |
    |                   |                     | ``printf("%lu")``.             |
    +-------------------+---------------------+--------------------------------+
    | :attr:`%lld`      | long long           | Exactly equivalent to          |
    |                   |                     | ``printf("%lld")``.            |
    +-------------------+---------------------+--------------------------------+
+   | :attr:`%lli`      | long long           | Exactly equivalent to          |
+   |                   |                     | ``printf("%lli")``.            |
+   +-------------------+---------------------+--------------------------------+
    | :attr:`%llu`      | unsigned long long  | Exactly equivalent to          |
    |                   |                     | ``printf("%llu")``.            |
    +-------------------+---------------------+--------------------------------+
    | :attr:`%zd`       | Py_ssize_t          | Exactly equivalent to          |
    |                   |                     | ``printf("%zd")``.             |
    +-------------------+---------------------+--------------------------------+
+   | :attr:`%zi`       | Py_ssize_t          | Exactly equivalent to          |
+   |                   |                     | ``printf("%zi")``.             |
+   +-------------------+---------------------+--------------------------------+
    | :attr:`%zu`       | size_t              | Exactly equivalent to          |
    |                   |                     | ``printf("%zu")``.             |
    +-------------------+---------------------+--------------------------------+
@@ -322,6 +331,9 @@ APIs:
    .. versionchanged:: 3.2
       Support for ``"%lld"`` and ``"%llu"`` added.
 
+   .. versionchanged:: 3.3
+      Support for ``"%li"``, ``"%lli"`` and ``"%zi"`` added.
+
 
 .. c:function:: PyObject* PyUnicode_FromFormatV(const char *format, va_list vargs)
 

Doc/contents.rst

@@ -11,7 +11,7 @@
    library/index.rst
    extending/index.rst
    c-api/index.rst
-   distutils/index.rst
+   packaging/index.rst
    install/index.rst
    documenting/index.rst
    howto/index.rst

Doc/distutils/index.rst

@@ -14,6 +14,10 @@ the module developer's point of view, describing how to use the Distutils to
 make Python modules and extensions easily available to a wider audience with
 very little overhead for build/release/install mechanics.
 
+.. deprecated:: 3.3
+   :mod:`packaging` replaces Distutils.  See :ref:`packaging-index` and
+   :ref:`packaging-install-index`.
+
 .. toctree::
    :maxdepth: 2
    :numbered:
@@ -29,3 +33,10 @@ very little overhead for build/release/install mechanics.
    extending.rst
    commandref.rst
    apiref.rst
+
+Another document describes how to install modules and extensions packaged
+following the above guidelines:
+
+.. toctree::
+
+   install.rst

Doc/documenting/style.rst

@@ -14,9 +14,10 @@ document.
 Use of whitespace
 -----------------
 
-All reST files use an indentation of 3 spaces.  The maximum line length is 80
-characters for normal text, but tables, deeply indented code samples and long
-links may extend beyond that.
+All reST files use an indentation of 3 spaces; no tabs are allowed.  The
+maximum line length is 80 characters for normal text, but tables, deeply
+indented code samples and long links may extend beyond that.  Code example
+bodies should use normal Python 4-space indentation.
 
 Make generous use of blank lines where applicable; they help grouping things
 together.

Doc/faq/general.rst

@@ -166,7 +166,7 @@ and run out of the box on most UNIX platforms.
 
 .. XXX update link once the dev faq is relocated
 
-Consult the `Developer FAQ <http://www.python.org/dev/faq/>`__ for more
+Consult the `Developer FAQ <http://docs.python.org/devguide/faq>`__ for more
 information on getting the source code and compiling it.
 
 
@@ -224,7 +224,7 @@ news is available.
 .. XXX update link once the dev faq is relocated
 
 You can also access the development version of Python through Subversion.  See
-http://www.python.org/dev/faq/ for details.
+http://docs.python.org/devguide/faq for details.
 
 
 How do I submit bug reports and patches for Python?
@@ -242,7 +242,7 @@ report bugs to Python, you can obtain your Roundup password through Roundup's
 .. XXX adapt link to dev guide
 
 For more information on how Python is developed, consult `the Python Developer's
-Guide <http://python.org/dev/>`_.
+Guide <http://docs.python.org/devguide/>`_.
 
 
 Are there any published articles about Python that I can reference?

Doc/glossary.rst

@@ -31,7 +31,7 @@ Glossary
       providing a way to define interfaces when other techniques like
       :func:`hasattr` would be clumsy or subtly wrong (for example with
       :ref:`magic methods <special-lookup>`). Python comes with many built-in ABCs for
-      data structures (in the :mod:`collections` module), numbers (in the
+      data structures (in the :mod:`collections.abc` module), numbers (in the
       :mod:`numbers` module), streams (in the :mod:`io` module), import finders
       and loaders (in the :mod:`importlib.abc` module).  You can create your own
       ABCs with the :mod:`abc` module.
@@ -582,6 +582,14 @@ Glossary
       an :term:`expression` or a one of several constructs with a keyword, such
       as :keyword:`if`, :keyword:`while` or :keyword:`for`.
 
+   struct sequence
+      A tuple with named elements. Struct sequences expose an interface similiar
+      to :term:`named tuple` in that elements can either be accessed either by
+      index or as an attribute. However, they do not have any of the named tuple
+      methods like :meth:`~collections.somenamedtuple._make` or
+      :meth:`~collections.somenamedtuple._asdict`. Examples of struct sequences
+      include :data:`sys.float_info` and the return value of :func:`os.stat`.
+
    triple-quoted string
       A string which is bound by three instances of either a quotation mark
       (") or an apostrophe (').  While they don't provide any functionality

Doc/howto/sockets.rst

@@ -23,8 +23,8 @@ It's not really a tutorial - you'll still have work to do in getting things
 working. It doesn't cover the fine points (and there are a lot of them), but I
 hope it will give you enough background to begin using them decently.
 
-I'm only going to talk about INET sockets, but they account for at least 99% of
-the sockets in use. And I'll only talk about STREAM sockets - unless you really
+I'm only going to talk about INET (i.e. IPv4) sockets, but they account for at least 99% of
+the sockets in use. And I'll only talk about STREAM (i.e. TCP) sockets - unless you really
 know what you're doing (in which case this HOWTO isn't for you!), you'll get
 better behavior and performance from a STREAM socket than anything else. I will
 try to clear up the mystery of what a socket is, as well as some hints on how to
@@ -208,10 +208,10 @@ length message::
                totalsent = totalsent + sent
 
        def myreceive(self):
-           msg = ''
+           msg = b''
            while len(msg) < MSGLEN:
                chunk = self.sock.recv(MSGLEN-len(msg))
-               if chunk == '':
+               if chunk == b'':
                    raise RuntimeError("socket connection broken")
                msg = msg + chunk
            return msg
@@ -371,12 +371,6 @@ have created a new socket to ``connect`` to someone else, put it in the
 potential_writers list. If it shows up in the writable list, you have a decent
 chance that it has connected.
 
-One very nasty problem with ``select``: if somewhere in those input lists of
-sockets is one which has died a nasty death, the ``select`` will fail. You then
-need to loop through every single damn socket in all those lists and do a
-``select([sock],[],[],0)`` until you find the bad one. That timeout of 0 means
-it won't take long, but it's ugly.
-
 Actually, ``select`` can be handy even with blocking sockets. It's one way of
 determining whether you will block - the socket returns as readable when there's
 something in the buffers.  However, this still doesn't help with the problem of
@@ -386,32 +380,6 @@ determining whether the other end is done, or just busy with something else.
 files. Don't try this on Windows. On Windows, ``select`` works with sockets
 only. Also note that in C, many of the more advanced socket options are done
 differently on Windows. In fact, on Windows I usually use threads (which work
-very, very well) with my sockets. Face it, if you want any kind of performance,
-your code will look very different on Windows than on Unix.
-
-
-Performance
------------
+very, very well) with my sockets.
 
-There's no question that the fastest sockets code uses non-blocking sockets and
-select to multiplex them. You can put together something that will saturate a
-LAN connection without putting any strain on the CPU. The trouble is that an app
-written this way can't do much of anything else - it needs to be ready to
-shuffle bytes around at all times.
-
-Assuming that your app is actually supposed to do something more than that,
-threading is the optimal solution, (and using non-blocking sockets will be
-faster than using blocking sockets). Unfortunately, threading support in Unixes
-varies both in API and quality. So the normal Unix solution is to fork a
-subprocess to deal with each connection. The overhead for this is significant
-(and don't do this on Windows - the overhead of process creation is enormous
-there). It also means that unless each subprocess is completely independent,
-you'll need to use another form of IPC, say a pipe, or shared memory and
-semaphores, to communicate between the parent and child processes.
-
-Finally, remember that even though blocking sockets are somewhat slower than
-non-blocking, in many cases they are the "right" solution. After all, if your
-app is driven by the data it receives over a socket, there's not much sense in
-complicating the logic just so your app can wait on ``select`` instead of
-``recv``.
 

Doc/install/pysetup-config.rst

+.. _packaging-pysetup-config:
+
+=====================
+Pysetup Configuration
+=====================
+
+Pysetup supports two configuration files: :file:`.pypirc` and :file:`packaging.cfg`.
+
+.. FIXME integrate with configfile instead of duplicating
+
+Configuring indexes
+-------------------
+
+You can configure additional indexes in :file:`.pypirc` to be used for index-related
+operations. By default, all configured index-servers and package-servers will be used
+in an additive fashion. To limit operations to specific indexes, use the :option:`--index`
+and :option:`--package-server options`::
+
+  $ pysetup install --index pypi --package-server django some.project
+
+Adding indexes to :file:`.pypirc`::
+
+  [packaging]
+  index-servers =
+      pypi
+      other
+
+  package-servers =
+      django
+
+  [pypi]
+      repository: <repository-url>
+      username: <username>
+      password: <password>
+
+  [other]
+      repository: <repository-url>
+      username: <username>
+      password: <password>
+
+  [django]
+      repository: <repository-url>
+      username: <username>
+      password: <password>

Doc/install/pysetup-servers.rst

+.. _packaging-pysetup-servers:
+
+===============
+Package Servers
+===============
+
+Pysetup supports installing Python packages from *Package Servers* in addition
+to PyPI indexes and mirrors.
+
+Package Servers are simple directory listings of Python distributions. Directories
+can be served via HTTP or a local file system. This is useful when you want to
+dump source distributions in a directory and not worry about the full index structure.
+
+Serving distributions from Apache
+---------------------------------
+::
+
+   $ mkdir -p /var/www/html/python/distributions
+   $ cp *.tar.gz /var/www/html/python/distributions/
+
+   <VirtualHost python.example.org:80>
+       ServerAdmin webmaster@domain.com
+       DocumentRoot "/var/www/html/python"
+       ServerName python.example.org
+       ErrorLog logs/python.example.org-error.log
+       CustomLog logs/python.example.org-access.log common
+       Options Indexes FollowSymLinks MultiViews
+       DirectoryIndex index.html index.htm
+
+       <Directory "/var/www/html/python/distributions">
+           Options Indexes FollowSymLinks MultiViews
+           Order allow,deny
+           Allow from all
+       </Directory>
+   </VirtualHost>
+
+Add the Apache based distribution server to :file:`.pypirc`::
+
+   [packaging]
+   package-servers =
+       apache
+
+   [apache]
+   repository: http://python.example.org/distributions/
+
+
+Serving distributions from a file system
+----------------------------------------
+::
+
+   $ mkdir -p /data/python/distributions
+   $ cp *.tar.gz /data/python/distributions/
+
+Add the directory to :file:`.pypirc`::
+
+   [packaging]
+   package-servers =
+       local
+
+   [local]
+   repository: file:///data/python/distributions/

Doc/install/pysetup.rst

+.. _packaging-pysetup:
+
+================
+Pysetup Tutorial
+================
+
+Getting started
+---------------
+
+Pysetup is a simple script that supports the following features:
+
+- install, remove, list, and verify Python packages;
+- search for available packages on PyPI or any *Simple Index*;
+- verify installed packages (md5sum, installed files, version).
+
+
+Finding out what's installed
+----------------------------
+
+Pysetup makes it easy to find out what Python packages are installed::
+
+   $ pysetup search virtualenv
+   virtualenv 1.6 at /opt/python3.3/lib/python3.3/site-packages/virtualenv-1.6-py3.3.egg-info
+
+   $ pysetup search --all
+   pyverify 0.8.1 at /opt/python3.3/lib/python3.3/site-packages/pyverify-0.8.1.dist-info
+   virtualenv 1.6 at /opt/python3.3/lib/python3.3/site-packages/virtualenv-1.6-py3.3.egg-info
+   wsgiref 0.1.2 at /opt/python3.3/lib/python3.3/wsgiref.egg-info
+   ...
+
+
+Installing a distribution
+-------------------------
+
+Pysetup can install a Python project from the following sources:
+
+- PyPI and Simple Indexes;
+- source directories containing a valid :file:`setup.py` or :file:`setup.cfg`;
+- distribution source archives (:file:`project-1.0.tar.gz`, :file:`project-1.0.zip`);
+- HTTP (http://host/packages/project-1.0.tar.gz).
+
+
+Installing from PyPI and Simple Indexes::
+
+   $ pysetup install project
+   $ pysetup install project==1.0
+
+Installing from a distribution source archive::
+
+   $ pysetup install project-1.0.tar.gz
+
+Installing from a source directory containing a valid :file:`setup.py` or
+:file:`setup.cfg`::
+
+   $ cd path/to/source/directory
+   $ pysetup install
+
+   $ pysetup install path/to/source/directory
+
+Installing from HTTP::
+
+   $ pysetup install http://host/packages/project-1.0.tar.gz
+
+
+Retrieving metadata
+-------------------
+
+You can gather metadata from two sources, a project's source directory or an
+installed distribution. The `pysetup metadata` command can retrieve one or
+more metadata fields using the `-f` option and a metadata field as the
+argument. ::
+
+   $ pysetup metadata virtualenv -f version -f name
+   Version:
+       1.6
+   Name:
+       virtualenv
+
+   $ pysetup metadata virtualenv --all
+   Metadata-Version:
+       1.0
+   Name:
+       virtualenv
+   Version:
+       1.6
+   Platform:
+       UNKNOWN
+   Summary:
+       Virtual Python Environment builder
+   ...
+
+.. seealso::
+
+   There are three metadata versions, 1.0, 1.1, and 1.2. The following PEPs
+   describe specifics of the field names, and their semantics and usage.  1.0
+   :PEP:`241`, 1.1 :PEP:`314`, and 1.2 :PEP:`345`
+
+
+Removing a distribution
+-----------------------
+
+You can remove one or more installed distributions using the `pysetup remove`
+command::
+
+   $ pysetup remove virtualenv
+   removing 'virtualenv':
+     /opt/python3.3/lib/python3.3/site-packages/virtualenv-1.6-py3.3.egg-info/dependency_links.txt
+     /opt/python3.3/lib/python3.3/site-packages/virtualenv-1.6-py3.3.egg-info/entry_points.txt
+     /opt/python3.3/lib/python3.3/site-packages/virtualenv-1.6-py3.3.egg-info/not-zip-safe
+     /opt/python3.3/lib/python3.3/site-packages/virtualenv-1.6-py3.3.egg-info/PKG-INFO
+     /opt/python3.3/lib/python3.3/site-packages/virtualenv-1.6-py3.3.egg-info/SOURCES.txt
+     /opt/python3.3/lib/python3.3/site-packages/virtualenv-1.6-py3.3.egg-info/top_level.txt
+   Proceed (y/n)? y
+   success: removed 6 files and 1 dirs
+
+The optional '-y' argument auto confirms, skipping the conformation prompt::
+
+  $ pysetup remove virtualenv -y
+
+
+Getting help
+------------
+
+All pysetup actions take the `-h` and `--help` options which prints the commands
+help string to stdout. ::
+
+   $ pysetup remove -h
+   Usage: pysetup remove dist [-y]
+      or: pysetup remove --help
+
+   Uninstall a Python package.
+
+   positional arguments:
+      dist  installed distribution name
+
+   optional arguments:
+      -y  auto confirm package removal
+
+Getting a list of all pysetup actions and global options::
+
+   $ pysetup --help
+   Usage: pysetup [options] action [action_options]
+
+   Actions:
+       run: Run one or several commands
+       metadata: Display the metadata of a project
+       install: Install a project
+       remove: Remove a project
+       search: Search for a project
+       graph: Display a graph
+       create: Create a Project
+
+   To get more help on an action, use:
+
+       pysetup action --help
+
+   Global options:
+       --verbose (-v)  run verbosely (default)
+       --quiet (-q)    run quietly (turns verbosity off)
+       --dry-run (-n)  don't actually do anything
+       --help (-h)     show detailed help message
+       --no-user-cfg   ignore pydistutils.cfg in your home directory
+       --version       Display the version

Doc/library/_thread.rst

@@ -35,6 +35,9 @@ It defines the following constants and functions:
 
    Raised on thread-specific errors.
 
+   .. versionchanged:: 3.3
+      This is now a synonym of the built-in :exc:`RuntimeError`.
+
 
 .. data:: LockType
 

Doc/library/abc.rst

@@ -18,7 +18,7 @@ regarding a type hierarchy for numbers based on ABCs.)
 
 The :mod:`collections` module has some concrete classes that derive from
 ABCs; these can, of course, be further derived. In addition the
-:mod:`collections` module has some ABCs that can be used to test whether
+:mod:`collections.abc` submodule has some ABCs that can be used to test whether
 a class or instance provides a particular interface, for example, is it
 hashable or a mapping.
 
@@ -55,6 +55,9 @@ This module provides the following class:
         assert issubclass(tuple, MyABC)
         assert isinstance((), MyABC)
 
+      .. versionchanged:: 3.3
+         Returns the registered subclass, to allow usage as a class decorator.
+
    You can also override this method in an abstract base class:
 
    .. method:: __subclasshook__(subclass)

Doc/library/argparse.rst

@@ -362,16 +362,16 @@ formatter_class
 ^^^^^^^^^^^^^^^
 
 :class:`ArgumentParser` objects allow the help formatting to be customized by
-specifying an alternate formatting class.  Currently, there are three such
+specifying an alternate formatting class.  Currently, there are four such
 classes:
 
 .. class:: RawDescriptionHelpFormatter
            RawTextHelpFormatter
            ArgumentDefaultsHelpFormatter
+           MetavarTypeHelpFormatter
 
-The first two allow more control over how textual descriptions are displayed,
-while the last automatically adds information about argument default values.
-
+:class:`RawDescriptionHelpFormatter` and :class:`RawTextHelpFormatter` give
+more control over how textual descriptions are displayed.
 By default, :class:`ArgumentParser` objects line-wrap the description_ and
 epilog_ texts in command-line help messages::
 
@@ -395,7 +395,7 @@ epilog_ texts in command-line help messages::
    likewise for this epilog whose whitespace will be cleaned up and whose words
    will be wrapped across a couple lines
 
-Passing :class:`~argparse.RawDescriptionHelpFormatter` as ``formatter_class=``
+Passing :class:`RawDescriptionHelpFormatter` as ``formatter_class=``
 indicates that description_ and epilog_ are already correctly formatted and
 should not be line-wrapped::
 
@@ -421,11 +421,11 @@ should not be line-wrapped::
    optional arguments:
     -h, --help  show this help message and exit
 
-:class:`RawTextHelpFormatter` maintains whitespace for all sorts of help text
+:class:`RawTextHelpFormatter` maintains whitespace for all sorts of help text,
 including argument descriptions.
 
-The other formatter class available, :class:`ArgumentDefaultsHelpFormatter`,
-will add information about the default value of each of the arguments::
+:class:`ArgumentDefaultsHelpFormatter` automatically adds information about
+default values to each of the argument help messages::
 
    >>> parser = argparse.ArgumentParser(
    ...     prog='PROG',
@@ -442,6 +442,25 @@ will add information about the default value of each of the arguments::
     -h, --help  show this help message and exit
     --foo FOO   FOO! (default: 42)
 
+:class:`MetavarTypeHelpFormatter` uses the name of the type_ argument for each
+argument as as the display name for its values (rather than using the dest_
+as the regular formatter does)::
+
+   >>> parser = argparse.ArgumentParser(
+   ...     prog='PROG',
+   ...     formatter_class=argparse.MetavarTypeHelpFormatter)
+   >>> parser.add_argument('--foo', type=int)
+   >>> parser.add_argument('bar', type=float)
+   >>> parser.print_help()
+   usage: PROG [-h] [--foo int] float
+
+   positional arguments:
+     float
+
+   optional arguments:
+     -h, --help  show this help message and exit
+     --foo int
+
 
 conflict_handler
 ^^^^^^^^^^^^^^^^

Doc/library/ast.rst

@@ -96,8 +96,8 @@ Node classes
 Abstract Grammar
 ----------------
 
-The module defines a string constant ``__version__`` which is the decimal
-Subversion revision number of the file shown below.
+The module defines a string constant ``__version__`` which is the Mercurial
+revision of the file shown below.
 
 The abstract grammar is currently defined as follows:
 

Doc/library/asyncore.rst

@@ -184,12 +184,14 @@ any that have been added to the map during asynchronous service) is closed.
    Most of these are nearly identical to their socket partners.
 
 
-   .. method:: create_socket(family, type)
+   .. method:: create_socket(family=socket.AF_INET, type=socket.SOCK_STREAM)
 
       This is identical to the creation of a normal socket, and will use the
       same options for creation.  Refer to the :mod:`socket` documentation for
       information on creating sockets.
 
+  .. versionchanged:: 3.3 family and type arguments can be omitted.
+
 
    .. method:: connect(address)
 
@@ -280,7 +282,7 @@ implement its socket handling::
 
        def __init__(self, host, path):
            asyncore.dispatcher.__init__(self)
-           self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
+           self.create_socket()
            self.connect( (host, 80) )
            self.buffer = bytes('GET %s HTTP/1.0\r\nHost: %s\r\n\r\n' %
                                (path, host), 'ascii')
@@ -327,7 +329,7 @@ connections and dispatches the incoming connections to a handler::
 
         def __init__(self, host, port):
             asyncore.dispatcher.__init__(self)
-            self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
+            self.create_socket()
             self.set_reuse_addr()
             self.bind((host, port))
             self.listen(5)

Doc/library/atexit.rst

@@ -62,7 +62,8 @@ automatically when the program terminates without relying on the application
 making an explicit call into this module at termination. ::
 
    try:
-       _count = int(open("/tmp/counter").read())
+       with open("/tmp/counter") as infile:
+           _count = int(infile.read())
    except IOError:
        _count = 0
 
@@ -71,7 +72,8 @@ making an explicit call into this module at termination. ::
        _count = _count + n
 
    def savecounter():
-       open("/tmp/counter", "w").write("%d" % _count)
+       with open("/tmp/counter", "w") as outfile:
+           outfile.write("%d" % _count)
 
    import atexit
    atexit.register(savecounter)

Doc/library/cmd.rst

@@ -282,8 +282,8 @@ immediate playback::
         def do_playback(self, arg):
             'Playback commands from a file:  PLAYBACK rose.cmd'
             self.close()
-            cmds = open(arg).read().splitlines()
-            self.cmdqueue.extend(cmds)
+            with open(arg) as f:
+                self.cmdqueue.extend(f.read().splitlines())
         def precmd(self, line):
             line = line.lower()
             if self.file and 'playback' not in line:

Doc/library/codecs.rst

@@ -458,7 +458,8 @@ define in order to be compatible with the Python codec registry.
 
    .. method:: reset()
 
-      Reset the encoder to the initial state.
+      Reset the encoder to the initial state. The output is discarded: call
+      ``.encode('', final=True)`` to reset the encoder and to get the output.
 
 
 .. method:: IncrementalEncoder.getstate()
@@ -904,6 +905,15 @@ is meant to be exhaustive. Notice that spelling alternatives that only differ in
 case or use a hyphen instead of an underscore are also valid aliases; therefore,
 e.g. ``'utf-8'`` is a valid alias for the ``'utf_8'`` codec.
 
+.. impl-detail::
+
+   Some common encodings can bypass the codecs lookup machinery to
+   improve performance.  These optimization opportunities are only
+   recognized by CPython for a limited set of aliases: utf-8, utf8,
+   latin-1, latin1, iso-8859-1, mbcs (Windows only), ascii, utf-16,
+   and utf-32.  Using alternative spellings for these encodings may
+   result in slower execution.
+
 Many of the character sets support the same languages. They vary in individual
 characters (e.g. whether the EURO SIGN is supported or not), and in the
 assignment of characters to code positions. For the European languages in

Doc/library/collections.abc.rst

+:mod:`collections.abc` --- Abstract Base Classes for Containers
+===============================================================
+
+.. module:: collections.abc
+   :synopsis: Abstract base classes for containers
+.. moduleauthor:: Raymond Hettinger <python at rcn.com>
+.. sectionauthor:: Raymond Hettinger <python at rcn.com>
+
+.. testsetup:: *
+
+   from collections import *
+   import itertools
+   __name__ = '<doctest>'
+
+**Source code:** :source:`Lib/collections/abc.py`
+
+--------------
+
+This module provides :term:`abstract base classes <abstract base class>` that
+can be used to test whether a class provides a particular interface; for
+example, whether it is hashable or whether it is a mapping.
+
+.. versionchanged:: 3.3
+   Formerly, this module was part of the :mod:`collections` module.
+
+.. _collections-abstract-base-classes:
+
+Collections Abstract Base Classes
+---------------------------------
+
+The collections module offers the following :term:`ABCs <abstract base class>`:
+
+=========================  =====================  ======================  ====================================================
+ABC                        Inherits from          Abstract Methods        Mixin Methods
+=========================  =====================  ======================  ====================================================
+:class:`Container`                                ``__contains__``
+:class:`Hashable`                                 ``__hash__``
+:class:`Iterable`                                 ``__iter__``
+:class:`Iterator`          :class:`Iterable`      ``__next__``            ``__iter__``
+:class:`Sized`                                    ``__len__``
+:class:`Callable`                                 ``__call__``
+
+:class:`Sequence`          :class:`Sized`,        ``__getitem__``         ``__contains__``, ``__iter__``, ``__reversed__``,
+                           :class:`Iterable`,                             ``index``, and ``count``
+                           :class:`Container`
+
+:class:`MutableSequence`   :class:`Sequence`      ``__setitem__``,        Inherited :class:`Sequence` methods and
+                                                  ``__delitem__``,        ``append``, ``reverse``, ``extend``, ``pop``,
+                                                  ``insert``              ``remove``, ``clear``, and ``__iadd__``
+
+:class:`Set`               :class:`Sized`,                                ``__le__``, ``__lt__``, ``__eq__``, ``__ne__``,
+                           :class:`Iterable`,                             ``__gt__``, ``__ge__``, ``__and__``, ``__or__``,
+                           :class:`Container`                             ``__sub__``, ``__xor__``, and ``isdisjoint``
+
+:class:`MutableSet`        :class:`Set`           ``add``,                Inherited :class:`Set` methods and
+                                                  ``discard``             ``clear``, ``pop``, ``remove``, ``__ior__``,
+                                                                          ``__iand__``, ``__ixor__``, and ``__isub__``
+
+:class:`Mapping`           :class:`Sized`,        ``__getitem__``         ``__contains__``, ``keys``, ``items``, ``values``,
+                           :class:`Iterable`,                             ``get``, ``__eq__``, and ``__ne__``
+                           :class:`Container`
+
+:class:`MutableMapping`    :class:`Mapping`       ``__setitem__``,        Inherited :class:`Mapping` methods and
+                                                  ``__delitem__``         ``pop``, ``popitem``, ``clear``, ``update``,
+                                                                          and ``setdefault``
+
+
+:class:`MappingView`       :class:`Sized`                                 ``__len__``
+:class:`ItemsView`         :class:`MappingView`,                          ``__contains__``,
+                           :class:`Set`                                   ``__iter__``
+:class:`KeysView`          :class:`MappingView`,                          ``__contains__``,
+                           :class:`Set`                                   ``__iter__``
+:class:`ValuesView`        :class:`MappingView`                           ``__contains__``, ``__iter__``
+=========================  =====================  ======================  ====================================================
+
+
+.. class:: Container
+           Hashable
+           Sized
+           Callable
+
+   ABCs for classes that provide respectively the methods :meth:`__contains__`,
+   :meth:`__hash__`, :meth:`__len__`, and :meth:`__call__`.
+
+.. class:: Iterable
+
+   ABC for classes that provide the :meth:`__iter__` method.
+   See also the definition of :term:`iterable`.
+
+.. class:: Iterator
+
+   ABC for classes that provide the :meth:`__iter__` and :meth:`next` methods.
+   See also the definition of :term:`iterator`.
+
+.. class:: Sequence
+           MutableSequence
+
+   ABCs for read-only and mutable :term:`sequences <sequence>`.
+
+.. class:: Set
+           MutableSet
+
+   ABCs for read-only and mutable sets.
+
+.. class:: Mapping
+           MutableMapping
+
+   ABCs for read-only and mutable :term:`mappings <mapping>`.
+
+.. class:: MappingView
+           ItemsView
+           KeysView
+           ValuesView
+
+   ABCs for mapping, items, keys, and values :term:`views <view>`.
+
+
+These ABCs allow us to ask classes or instances if they provide
+particular functionality, for example::
+
+    size = None
+    if isinstance(myvar, collections.Sized):
+        size = len(myvar)
+
+Several of the ABCs are also useful as mixins that make it easier to develop
+classes supporting container APIs.  For example, to write a class supporting
+the full :class:`Set` API, it only necessary to supply the three underlying
+abstract methods: :meth:`__contains__`, :meth:`__iter__`, and :meth:`__len__`.
+The ABC supplies the remaining methods such as :meth:`__and__` and
+:meth:`isdisjoint` ::
+
+    class ListBasedSet(collections.Set):
+         ''' Alternate set implementation favoring space over speed
+             and not requiring the set elements to be hashable. '''
+         def __init__(self, iterable):
+             self.elements = lst = []
+             for value in iterable:
+                 if value not in lst:
+                     lst.append(value)
+         def __iter__(self):
+             return iter(self.elements)
+         def __contains__(self, value):
+             return value in self.elements
+         def __len__(self):
+             return len(self.elements)
+
+    s1 = ListBasedSet('abcdef')
+    s2 = ListBasedSet('defghi')
+    overlap = s1 & s2            # The __and__() method is supported automatically
+
+Notes on using :class:`Set` and :class:`MutableSet` as a mixin:
+
+(1)
+   Since some set operations create new sets, the default mixin methods need
+   a way to create new instances from an iterable. The class constructor is
+   assumed to have a signature in the form ``ClassName(iterable)``.
+   That assumption is factored-out to an internal classmethod called
+   :meth:`_from_iterable` which calls ``cls(iterable)`` to produce a new set.
+   If the :class:`Set` mixin is being used in a class with a different
+   constructor signature, you will need to override :meth:`_from_iterable`
+   with a classmethod that can construct new instances from
+   an iterable argument.
+
+(2)
+   To override the comparisons (presumably for speed, as the
+   semantics are fixed), redefine :meth:`__le__` and
+   then the other operations will automatically follow suit.
+
+(3)
+   The :class:`Set` mixin provides a :meth:`_hash` method to compute a hash value
+   for the set; however, :meth:`__hash__` is not defined because not all sets
+   are hashable or immutable.  To add set hashabilty using mixins,
+   inherit from both :meth:`Set` and :meth:`Hashable`, then define
+   ``__hash__ = Set._hash``.
+
+.. seealso::
+
+   * `OrderedSet recipe <http://code.activestate.com/recipes/576694/>`_ for an
+     example built on :class:`MutableSet`.
+
+   * For more about ABCs, see the :mod:`abc` module and :pep:`3119`.

Doc/library/concurrent.futures.rst

@@ -169,6 +169,12 @@ to a :class:`ProcessPoolExecutor` will result in deadlock.
    of at most *max_workers* processes.  If *max_workers* is ``None`` or not
    given, it will default to the number of processors on the machine.
 
+   .. versionchanged:: 3.3
+      When one of the worker processes terminates abruptly, a
+      :exc:`BrokenProcessPool` error is now raised.  Previously, behaviour
+      was undefined but operations on the executor or its futures would often
+      freeze or deadlock.
+
 
 .. _processpoolexecutor-example:
 
@@ -369,3 +375,16 @@ Module Functions
    :pep:`3148` -- futures - execute computations asynchronously
       The proposal which described this feature for inclusion in the Python
       standard library.
+
+
+Exception classes
+-----------------
+
+.. exception:: BrokenProcessPool
+
+   Derived from :exc:`RuntimeError`, this exception class is raised when
+   one of the workers of a :class:`ProcessPoolExecutor` has terminated
+   in a non-clean fashion (for example, if it was killed from the outside).
+
+   .. versionadded:: 3.3
+

Doc/library/csv.rst

@@ -11,15 +11,15 @@
    pair: data; tabular
 
 The so-called CSV (Comma Separated Values) format is the most common import and
-export format for spreadsheets and databases.  There is no "CSV standard", so
-the format is operationally defined by the many applications which read and
-write it.  The lack of a standard means that subtle differences often exist in
-the data produced and consumed by different applications.  These differences can
-make it annoying to process CSV files from multiple sources.  Still, while the
-delimiters and quoting characters vary, the overall format is similar enough
-that it is possible to write a single module which can efficiently manipulate
-such data, hiding the details of reading and writing the data from the
-programmer.
+export format for spreadsheets and databases.  CSV format was used for many
+years prior to attempts to describe the format in a standardized way in
+:rfc:`4180`.  The lack of a well-defined standard means that subtle differences
+often exist in the data produced and consumed by different applications.  These
+differences can make it annoying to process CSV files from multiple sources.
+Still, while the delimiters and quoting characters vary, the overall format is
+similar enough that it is possible to write a single module which can
+efficiently manipulate such data, hiding the details of reading and writing the
+data from the programmer.
 
 The :mod:`csv` module implements classes to read and write tabular data in CSV
 format.  It allows programmers to say, "write this data in the format preferred

Doc/library/datatypes.rst

@@ -21,6 +21,7 @@ The following modules are documented in this chapter:
    datetime.rst
    calendar.rst
    collections.rst
+   collections.abc.rst
    heapq.rst
    bisect.rst
    array.rst

Doc/library/datetime.rst

@@ -721,6 +721,22 @@ Other constructors, all class methods:
    It's common for this to be restricted to years in 1970 through 2038. See also
    :meth:`fromtimestamp`.
 
+   On the POSIX compliant platforms, ``utcfromtimestamp(timestamp)``
+   is equivalent to the following expression::
+
+     datetime(1970, 1, 1) + timedelta(seconds=timestamp)
+
+   There is no method to obtain the timestamp from a :class:`datetime`
+   instance, but POSIX timestamp corresponding to a :class:`datetime`
+   instance ``dt`` can be easily calculated as follows.  For a naive
+   ``dt``::
+
+      timestamp = (dt - datetime(1970, 1, 1)) / timedelta(seconds=1)
+
+   And for an aware ``dt``::
+
+      timestamp = (dt - datetime(1970, 1, 1, tzinfo=timezone.utc)) / timedelta(seconds=1)
+
 
 .. classmethod:: datetime.fromordinal(ordinal)
 
@@ -1561,11 +1577,12 @@ only EST (fixed offset -5 hours), or only EDT (fixed offset -4 hours)).
 :class:`timezone` Objects
 --------------------------
 
-A :class:`timezone` object represents a timezone that is defined by a
-fixed offset from UTC.  Note that objects of this class cannot be used
-to represent timezone information in the locations where different
-offsets are used in different days of the year or where historical
-changes have been made to civil time.
+The :class:`timezone` class is a subclass of :class:`tzinfo`, each
+instance of which represents a timezone defined by a fixed offset from
+UTC.  Note that objects of this class cannot be used to represent
+timezone information in the locations where different offsets are used
+in different days of the year or where historical changes have been
+made to civil time.
 
 
 .. class:: timezone(offset[, name])
@@ -1734,8 +1751,7 @@ format codes.
 |           | decimal number [00,99].        |       |
 +-----------+--------------------------------+-------+
 | ``%Y``    | Year with century as a decimal | \(5)  |
-|           | number [0001,9999] (strptime), |       |
-|           | [1000,9999] (strftime).        |       |
+|           | number [0001,9999].            |       |
 +-----------+--------------------------------+-------+
 | ``%z``    | UTC offset in the form +HHMM   | \(6)  |
 |           | or -HHMM (empty string if the  |       |
@@ -1769,10 +1785,7 @@ Notes:
    calculations when the day of the week and the year are specified.
 
 (5)
-   For technical reasons, :meth:`strftime` method does not support
-   dates before year 1000: ``t.strftime(format)`` will raise a
-   :exc:`ValueError` when ``t.year < 1000`` even if ``format`` does
-   not contain ``%Y`` directive.  The :meth:`strptime` method can
+   The :meth:`strptime` method can
    parse years in the full [1, 9999] range, but years < 1000 must be
    zero-filled to 4-digit width.
 
@@ -1780,6 +1793,10 @@ Notes:
       In previous versions, :meth:`strftime` method was restricted to
       years >= 1900.
 
+   .. versionchanged:: 3.3
+      In version 3.2, :meth:`strftime` method was restricted to
+      years >= 1000.
+
 (6)
    For example, if :meth:`utcoffset` returns ``timedelta(hours=-3, minutes=-30)``,
    ``%z`` is replaced with the string ``'-0330'``.

Doc/library/debug.rst

@@ -10,7 +10,8 @@ allowing you to identify bottlenecks in your programs.
 .. toctree::
 
    bdb.rst
+   faulthandler.rst
    pdb.rst
    profile.rst
    timeit.rst
-   trace.rst
\ No newline at end of file
+   trace.rst

Doc/library/difflib.rst

@@ -752,8 +752,8 @@ It is also contained in the Python source distribution, as
        # we're passing these as arguments to the diff function
        fromdate = time.ctime(os.stat(fromfile).st_mtime)
        todate = time.ctime(os.stat(tofile).st_mtime)
-       fromlines = open(fromfile, 'U').readlines()
-       tolines = open(tofile, 'U').readlines()
+       with open(fromlines) as fromf, open(tofile) as tof:
+           fromlines, tolines = list(fromf), list(tof)
 
        if options.u:
            diff = difflib.unified_diff(fromlines, tolines, fromfile, tofile,

Doc/library/email.rst

@@ -51,6 +51,7 @@ Contents of the :mod:`email` package documentation:
    email.message.rst
    email.parser.rst
    email.generator.rst
+   email.policy.rst
    email.mime.rst
    email.header.rst
    email.charset.rst

Doc/library/python.rst

@@ -25,4 +25,5 @@ overview:
    inspect.rst
    site.rst
    fpectl.rst
+   packaging.rst
    distutils.rst