Merge.

Doc/library/asyncio-eventloop.rst

@@ -47,8 +47,9 @@ Run an event loop
    Stop running the event loop.
 
    Every callback scheduled before :meth:`stop` is called will run.
-   Callback scheduled after :meth:`stop` is called won't.  However, those
-   callbacks will run if :meth:`run_forever` is called again later.
+   Callbacks scheduled after :meth:`stop` is called will not run.
+   However, those callbacks will run if :meth:`run_forever` is called
+   again later.
 
 .. method:: BaseEventLoop.is_closed()
 
@@ -58,13 +59,11 @@ Run an event loop
 
 .. method:: BaseEventLoop.close()
 
-   Close the event loop. The loop should not be running.
+   Close the event loop. The loop must not be running.
 
    This clears the queues and shuts down the executor, but does not wait for
    the executor to finish.
 
-   The event loop must not be running.
-
    This is idempotent and irreversible. No other methods should be called after
    this one.
 

Doc/library/asyncio-protocol.rst

@@ -459,7 +459,7 @@ The event loop is running twice. The
 example to raise an exception if the server is not listening, instead of
 having to write a short coroutine to handle the exception and stop the
 running loop. At :meth:`~BaseEventLoop.run_until_complete` exit, the loop is
-no more running, so there is no need to stop the loop in case of an error.
+no longer running, so there is no need to stop the loop in case of an error.
 
 Echo server
 -----------

Doc/library/asyncio-task.rst

@@ -261,7 +261,7 @@ Example combining a :class:`Future` and a :ref:`coroutine function
     print(future.result())
     loop.close()
 
-The coroutine function is responsible of the computation (which takes 1 second)
+The coroutine function is responsible for the computation (which takes 1 second)
 and it stores the result into the future. The
 :meth:`~BaseEventLoop.run_until_complete` method waits for the completion of
 the future.

Doc/library/asyncio.rst

@@ -39,7 +39,7 @@ Here is a more detailed list of the package contents:
   you absolutely, positively have to use a library that makes blocking
   I/O calls.
 
-Table of content:
+Table of contents:
 
 .. toctree::
    :maxdepth: 3
@@ -55,6 +55,6 @@ Table of content:
 
 .. seealso::
 
-   The :mod:`asyncio` module was designed in the :PEP:`3156`. For a
+   The :mod:`asyncio` module was designed in :PEP:`3156`. For a
    motivational primer on transports and protocols, see :PEP:`3153`.
 

Doc/library/asyncore.rst

@@ -216,6 +216,10 @@ any that have been added to the map during asynchronous service) is closed.
       empty bytes object implies that the channel has been closed from the
       other end.
 
+      Note that :meth:`recv` may raise :exc:`BlockingIOError` , even though
+      :func:`select.select` or :func:`select.poll` has reported the socket
+      ready for reading.
+
 
    .. method:: listen(backlog)
 

Doc/library/multiprocessing.rst

@@ -398,7 +398,7 @@ For example::
            print(res.get(timeout=1))             # prints "100"
 
            # make worker sleep for 10 secs
-           res = pool.apply_async(sleep, 10)
+           res = pool.apply_async(sleep, [10])
            print(res.get(timeout=1))             # raises multiprocessing.TimeoutError
 
        # exiting the 'with'-block has stopped the pool

Doc/library/os.rst

@@ -765,8 +765,14 @@ as internal buffering of data.
 
 .. function:: fstat(fd)
 
-   Return status for file descriptor *fd*, like :func:`~os.stat`.  As of Python
-   3.3, this is equivalent to ``os.stat(fd)``.
+   Get the status of the file descriptor *fd*. Return a :class:`stat_result`
+   object.
+
+   As of Python 3.3, this is equivalent to ``os.stat(fd)``.
+
+   .. seealso::
+
+      The :func:`stat` function.
 
    Availability: Unix, Windows.
 
@@ -1088,8 +1094,16 @@ or `the MSDN <http://msdn.microsoft.com/en-us/library/z0kc8e3z.aspx>`_ on Window
    All platforms support sockets as *out* file descriptor, and some platforms
    allow other types (e.g. regular file, pipe) as well.
 
+   Cross-platform applications should not use *headers*, *trailers* and *flags*
+   arguments.
+
    Availability: Unix.
 
+   .. note::
+
+      For a higher-level wrapper of :func:`sendfile`, see
+      :mod:`socket.socket.sendfile`.
+
    .. versionadded:: 3.3
 
 
@@ -1570,17 +1584,25 @@ features:
       Added support for specifying an open file descriptor for *path*.
 
 
-.. function:: lstat(path, *, dir_fd=None)
+.. function:: lstat(path, \*, dir_fd=None)
 
    Perform the equivalent of an :c:func:`lstat` system call on the given path.
-   Similar to :func:`~os.stat`, but does not follow symbolic links.  On
-   platforms that do not support symbolic links, this is an alias for
-   :func:`~os.stat`.  As of Python 3.3, this is equivalent to ``os.stat(path,
-   dir_fd=dir_fd, follow_symlinks=False)``.
+   Similar to :func:`~os.stat`, but does not follow symbolic links. Return a
+   :class:`stat_result` object.
+
+   On platforms that do not support symbolic links, this is an alias for
+   :func:`~os.stat`.
+
+   As of Python 3.3, this is equivalent to ``os.stat(path, dir_fd=dir_fd,
+   follow_symlinks=False)``.
 
    This function can also support :ref:`paths relative to directory descriptors
    <dir_fd>`.
 
+   .. seealso::
+
+      The :func:`stat` function.
+
    .. versionchanged:: 3.2
       Added support for Windows 6.0 (Vista) symbolic links.
 
@@ -1847,49 +1869,116 @@ features:
       The *dir_fd* parameter.
 
 
-.. function:: stat(path, *, dir_fd=None, follow_symlinks=True)
-
-   Perform the equivalent of a :c:func:`stat` system call on the given path.
-   *path* may be specified as either a string or as an open file descriptor.
-   (This function normally follows symlinks; to stat a symlink add the argument
-   ``follow_symlinks=False``, or use :func:`lstat`.)
-
-   The return value is an object whose attributes correspond roughly
-   to the members of the :c:type:`stat` structure, namely:
-
-   * :attr:`st_mode` - protection bits,
-   * :attr:`st_ino` - inode number,
-   * :attr:`st_dev` - device,
-   * :attr:`st_nlink` - number of hard links,
-   * :attr:`st_uid` - user id of owner,
-   * :attr:`st_gid` - group id of owner,
-   * :attr:`st_size` - size of file, in bytes,
-   * :attr:`st_atime` - time of most recent access expressed in seconds,
-   * :attr:`st_mtime` - time of most recent content modification
-     expressed in seconds,
-   * :attr:`st_ctime` - platform dependent; time of most recent metadata
-     change on Unix, or the time of creation on Windows, expressed in seconds
-   * :attr:`st_atime_ns` - time of most recent access
-     expressed in nanoseconds as an integer,
-   * :attr:`st_mtime_ns` - time of most recent content modification
-     expressed in nanoseconds as an integer,
-   * :attr:`st_ctime_ns` - platform dependent; time of most recent metadata
-     change on Unix, or the time of creation on Windows,
-     expressed in nanoseconds as an integer
+.. function:: stat(path, \*, dir_fd=None, follow_symlinks=True)
 
-   On some Unix systems (such as Linux), the following attributes may also be
-   available:
+   Get the status of a file or a file descriptor. Perform the equivalent of a
+   :c:func:`stat` system call on the given path. *path* may be specified as
+   either a string or as an open file descriptor. Return a :class:`stat_result`
+   object.
 
-   * :attr:`st_blocks` - number of 512-byte blocks allocated for file
-   * :attr:`st_blksize` - filesystem blocksize for efficient file system I/O
-   * :attr:`st_rdev` - type of device if an inode device
-   * :attr:`st_flags` - user defined flags for file
+   This function normally follows symlinks; to stat a symlink add the argument
+   ``follow_symlinks=False``, or use :func:`lstat`.
 
-   On other Unix systems (such as FreeBSD), the following attributes may be
-   available (but may be only filled out if root tries to use them):
+   This function can support :ref:`specifying a file descriptor <path_fd>` and
+   :ref:`not following symlinks <follow_symlinks>`.
+
+   .. index:: module: stat
+
+   Example::
+
+      >>> import os
+      >>> statinfo = os.stat('somefile.txt')
+      >>> statinfo
+      os.stat_result(st_mode=33188, st_ino=7876932, st_dev=234881026,
+      st_nlink=1, st_uid=501, st_gid=501, st_size=264, st_atime=1297230295,
+      st_mtime=1297230027, st_ctime=1297230027)
+      >>> statinfo.st_size
+      264
+
+   Availability: Unix, Windows.
+
+   .. seealso::
+
+      :func:`fstat` and :func:`lstat` functions.
+
+   .. versionadded:: 3.3
+      Added the *dir_fd* and *follow_symlinks* arguments, specifying a file
+      descriptor instead of a path.
+
+
+.. class:: stat_result
+
+   Object whose attributes correspond roughly to the members of the
+   :c:type:`stat` structure. It is used for the result of :func:`os.stat`,
+   :func:`os.fstat` and :func:`os.lstat`.
+
+   Attributes:
+
+   .. attribute:: st_mode
+
+      File mode: file type and file mode bits (permissions).
+
+   .. attribute:: st_ino
+
+      Inode number.
+
+   .. attribute:: st_dev
+
+      Identifier of the device on which this file resides.
+
+   .. attribute:: st_nlink
+
+      Number of hard links.
+
+   .. attribute:: st_uid
+
+      User identifier of the file owner.
+
+   .. attribute:: st_gid
+
+      Group identifier of the file owner.
+
+   .. attribute:: st_size
+
+      Size of the file in bytes, if it is a regular file or a symbolic link.
+      The size of a symbolic link is the length of the pathname it contains,
+      without a terminating null byte.
+
+   Timestamps:
+
+   .. attribute:: st_atime
 
-   * :attr:`st_gen` - file generation number
-   * :attr:`st_birthtime` - time of file creation
+      Time of most recent access expressed in seconds.
+
+   .. attribute:: st_mtime
+
+      Time of most recent content modification expressed in seconds.
+
+   .. attribute:: st_ctime
+
+      Platform dependent:
+
+      * the time of most recent metadata change on Unix,
+      * the time of creation on Windows, expressed in seconds.
+
+   .. attribute:: st_atime_ns
+
+      Time of most recent access expressed in nanoseconds as an integer.
+
+   .. attribute:: st_mtime_ns
+
+      Time of most recent content modification expressed in nanoseconds as an
+      integer.
+
+   .. attribute:: st_ctime_ns
+
+      Platform dependent:
+
+      * the time of most recent metadata change on Unix,
+      * the time of creation on Windows, expressed in nanoseconds as an
+        integer.
+
+   See also the :func:`stat_float_times` function.
 
    .. note::
 
@@ -1899,6 +1988,7 @@ features:
       or FAT32 file systems, :attr:`st_mtime` has 2-second resolution, and
       :attr:`st_atime` has only 1-day resolution.  See your operating system
       documentation for details.
+
       Similarly, although :attr:`st_atime_ns`, :attr:`st_mtime_ns`,
       and :attr:`st_ctime_ns` are always expressed in nanoseconds, many
       systems do not provide nanosecond precision.  On systems that do
@@ -1908,41 +1998,68 @@ features:
       If you need the exact timestamps you should always use
       :attr:`st_atime_ns`, :attr:`st_mtime_ns`, and :attr:`st_ctime_ns`.
 
-   For backward compatibility, the return value of :func:`~os.stat` is also
-   accessible as a tuple of at least 10 integers giving the most important (and
-   portable) members of the :c:type:`stat` structure, in the order
-   :attr:`st_mode`, :attr:`st_ino`, :attr:`st_dev`, :attr:`st_nlink`,
-   :attr:`st_uid`, :attr:`st_gid`, :attr:`st_size`, :attr:`st_atime`,
-   :attr:`st_mtime`, :attr:`st_ctime`. More items may be added at the end by
-   some implementations.
+   On some Unix systems (such as Linux), the following attributes may also be
+   available:
 
-   This function can support :ref:`specifying a file descriptor <path_fd>` and
-   :ref:`not following symlinks <follow_symlinks>`.
+   .. attribute:: st_blocks
 
-   .. index:: module: stat
+      Number of 512-byte blocks allocated for file.
+      This may be smaller than :attr:`st_size`/512 when the file has holes.
 
-   The standard module :mod:`stat` defines functions and constants that are useful
-   for extracting information from a :c:type:`stat` structure. (On Windows, some
-   items are filled with dummy values.)
+   .. attribute:: st_blksize
 
-   Example::
+      "Preferred" blocksize for efficient file system I/O. Writing to a file in
+      smaller chunks may cause an inefficient read-modify-rewrite.
 
-      >>> import os
-      >>> statinfo = os.stat('somefile.txt')
-      >>> statinfo
-      posix.stat_result(st_mode=33188, st_ino=7876932, st_dev=234881026,
-      st_nlink=1, st_uid=501, st_gid=501, st_size=264, st_atime=1297230295,
-      st_mtime=1297230027, st_ctime=1297230027)
-      >>> statinfo.st_size
-      264
+   .. attribute:: st_rdev
 
-   Availability: Unix, Windows.
+      Type of device if an inode device.
+
+   .. attribute:: st_flags
+
+      User defined flags for file.
+
+   On other Unix systems (such as FreeBSD), the following attributes may be
+   available (but may be only filled out if root tries to use them):
+
+   .. attribute:: st_gen
+
+      File generation number.
+
+   .. attribute:: st_birthtime
+
+      Time of file creation.
+
+   On Mac OS systems, the following attributes may also be available:
+
+   .. attribute:: st_rsize
+
+      Real size of the file.
+
+   .. attribute:: st_creator
+
+      Creator of the file.
+
+   .. attribute:: st_type
+
+      File type.
+
+   The standard module :mod:`stat` defines functions and constants that are
+   useful for extracting information from a :c:type:`stat` structure. (On
+   Windows, some items are filled with dummy values.)
+
+   For backward compatibility, a :class:`stat_result` instance is also
+   accessible as a tuple of at least 10 integers giving the most important (and
+   portable) members of the :c:type:`stat` structure, in the order
+   :attr:`st_mode`, :attr:`st_ino`, :attr:`st_dev`, :attr:`st_nlink`,
+   :attr:`st_uid`, :attr:`st_gid`, :attr:`st_size`, :attr:`st_atime`,
+   :attr:`st_mtime`, :attr:`st_ctime`. More items may be added at the end by
+   some implementations. For compatibility with older Python versions,
+   accessing :class:`stat_result` as a tuple always returns integers.
 
    .. versionadded:: 3.3
-      Added the *dir_fd* and *follow_symlinks* arguments,
-      specifying a file descriptor instead of a path,
-      and the :attr:`st_atime_ns`, :attr:`st_mtime_ns`,
-      and :attr:`st_ctime_ns` members.
+      Added the :attr:`st_atime_ns`, :attr:`st_mtime_ns`, and
+      :attr:`st_ctime_ns` members.
 
 
 .. function:: stat_float_times([newvalue])
@@ -2727,10 +2844,27 @@ written in Python, such as a mail server's external command delivery program.
    Availability: Unix.
 
 
-.. function:: popen(...)
+.. function:: popen(command, mode='r', buffering=-1)
+
+   Open a pipe to or from *command*.  The return value is an open file object
+   connected to the pipe, which can be read or written depending on whether *mode*
+   is ``'r'`` (default) or ``'w'``. The *buffering* argument has the same meaning as
+   the corresponding argument to the built-in :func:`open` function. The
+   returned file object reads or writes text strings rather than bytes.
+
+   The ``close`` method returns :const:`None` if the subprocess exited
+   successfully, or the subprocess's return code if there was an
+   error. On POSIX systems, if the return code is positive it
+   represents the return value of the process left-shifted by one
+   byte.  If the return code is negative, the process was terminated
+   by the signal given by the negated value of the return code.  (For
+   example, the return value might be ``- signal.SIGKILL`` if the
+   subprocess was killed.)  On Windows systems, the return value
+   contains the signed integer return code from the child process.
 
-   Run child processes, returning opened pipes for communications.  These functions
-   are described in section :ref:`os-newstreams`.
+   This is implemented using :class:`subprocess.Popen`; see that class's
+   documentation for more powerful ways to manage and communicate with
+   subprocesses.
 
 
 .. function:: spawnl(mode, path, ...)

Doc/library/test.rst

@@ -461,7 +461,7 @@ The :mod:`test.support` module defines the following functions:
 .. function:: make_bad_fd()
 
    Create an invalid file descriptor by opening and closing a temporary file,
-   and returning its descripor.
+   and returning its descriptor.
 
 
 .. function:: import_module(name, deprecated=False)
@@ -554,6 +554,21 @@ The :mod:`test.support` module defines the following functions:
    run simultaneously, which is a problem for buildbots.
 
 
+.. function:: load_package_tests(pkg_dir, loader, standard_tests, pattern)
+
+   Generic implementation of the :mod:`unittest` ``load_tests`` protocol for
+   use in test packages.  *pkg_dir* is the root directory of the package;
+   *loader*, *standard_tests*, and *pattern* are the arguments expected by
+   ``load_tests``.  In simple cases, the test package's ``__init__.py``
+   can be the following::
+
+      import os
+      from test.support import load_package_tests
+
+      def load_tests(*args):
+          return load_package_tests(os.path.dirname(__file__), *args)
+
+
 The :mod:`test.support` module defines the following classes:
 
 .. class:: TransientResource(exc, **kwargs)

Lib/asynchat.py

@@ -115,6 +115,8 @@ class async_chat(asyncore.dispatcher):
 
         try:
             data = self.recv(self.ac_in_buffer_size)
+        except BlockingIOError:
+            return
         except OSError as why:
             self.handle_error()
             return

Lib/asyncio/base_events.py

@@ -270,9 +270,9 @@ class BaseEventLoop(events.AbstractEventLoop):
     def stop(self):
         """Stop running the event loop.
 
-        Every callback scheduled before stop() is called will run.
-        Callback scheduled after stop() is called won't.  However,
-        those callbacks will run if run_*() is called again later.
+        Every callback scheduled before stop() is called will run. Callbacks
+        scheduled after stop() is called will not run. However, those callbacks
+        will run if run_forever is called again later.
         """
         self.call_soon(_raise_stop_error)
 

Lib/asyncio/proactor_events.py

@@ -44,13 +44,9 @@ class _ProactorBasePipeTransport(transports._FlowControlMixin,
     def __repr__(self):
         info = [self.__class__.__name__, 'fd=%s' % self._sock.fileno()]
         if self._read_fut is not None:
-            ov = "pending" if self._read_fut.ov.pending else "completed"
-            info.append('read=%s' % ov)
+            info.append('read=%s' % self._read_fut)
         if self._write_fut is not None:
-            if self._write_fut.ov.pending:
-                info.append("write=pending=%s" % self._pending_write)
-            else:
-                info.append("write=completed")
+            info.append("write=%r" % self._write_fut)
         if self._buffer:
             bufsize = len(self._buffer)
             info.append('write_bufsize=%s' % bufsize)

Lib/asyncio/unix_events.py

@@ -74,7 +74,7 @@ class _UnixSelectorEventLoop(selector_events.BaseSelectorEventLoop):
             # event loop running in another thread cannot add a signal
             # handler.
             signal.set_wakeup_fd(self._csock.fileno())
-        except ValueError as exc:
+        except (ValueError, OSError) as exc:
             raise RuntimeError(str(exc))
 
         handle = events.Handle(callback, args, self)
@@ -93,7 +93,7 @@ class _UnixSelectorEventLoop(selector_events.BaseSelectorEventLoop):
             if not self._signal_handlers:
                 try:
                     signal.set_wakeup_fd(-1)
-                except ValueError as nexc:
+                except (ValueError, OSError) as nexc:
                     logger.info('set_wakeup_fd(-1) failed: %s', nexc)
 
             if exc.errno == errno.EINVAL:
@@ -138,7 +138,7 @@ class _UnixSelectorEventLoop(selector_events.BaseSelectorEventLoop):
         if not self._signal_handlers:
             try:
                 signal.set_wakeup_fd(-1)
-            except ValueError as exc:
+            except (ValueError, OSError) as exc:
                 logger.info('set_wakeup_fd(-1) failed: %s', exc)
 
         return True

Lib/asyncio/windows_events.py

@@ -38,42 +38,85 @@ class _OverlappedFuture(futures.Future):
 
     def __init__(self, ov, *, loop=None):
         super().__init__(loop=loop)
-        self.ov = ov
+        if self._source_traceback:
+            del self._source_traceback[-1]
+        self._ov = ov
 
     def __repr__(self):
         info = [self._state.lower()]
-        if self.ov.pending:
-            info.append('overlapped=pending')
-        else:
-            info.append('overlapped=completed')
+        if self._ov is not None:
+            state = 'pending' if self._ov.pending else 'completed'
+            info.append('overlapped=<%s, %#x>' % (state, self._ov.address))
         if self._state == futures._FINISHED:
             info.append(self._format_result())
         if self._callbacks:
             info.append(self._format_callbacks())
         return '<%s %s>' % (self.__class__.__name__, ' '.join(info))
 
-    def cancel(self):
+    def _cancel_overlapped(self):
+        if self._ov is None:
+            return
         try:
-            self.ov.cancel()
-        except OSError:
-            pass
+            self._ov.cancel()
+        except OSError as exc:
+            context = {
+                'message': 'Cancelling an overlapped future failed',
+                'exception': exc,
+                'future': self,
+            }
+            if self._source_traceback:
+                context['source_traceback'] = self._source_traceback
+            self._loop.call_exception_handler(context)
+        self._ov = None
+
+    def cancel(self):
+        self._cancel_overlapped()
         return super().cancel()
 
+    def set_exception(self, exception):
+        super().set_exception(exception)
+        self._cancel_overlapped()
+
 
 class _WaitHandleFuture(futures.Future):
     """Subclass of Future which represents a wait handle."""
 
-    def __init__(self, wait_handle, *, loop=None):
+    def __init__(self, handle, wait_handle, *, loop=None):
         super().__init__(loop=loop)
+        self._handle = handle
         self._wait_handle = wait_handle
 
-    def cancel(self):
-        super().cancel()
+    def _poll(self):
+        # non-blocking wait: use a timeout of 0 millisecond
+        return (_winapi.WaitForSingleObject(self._handle, 0) ==
+                _winapi.WAIT_OBJECT_0)
+
+    def __repr__(self):
+        info = [self._state.lower()]
+        if self._wait_handle:
+            state = 'pending' if self._poll() else 'completed'
+            info.append('wait_handle=<%s, %#x>' % (state, self._wait_handle))
+        info.append('handle=<%#x>' % self._handle)
+        if self._state == futures._FINISHED:
+            info.append(self._format_result())
+        if self._callbacks:
+            info.append(self._format_callbacks())
+        return '<%s %s>' % (self.__class__.__name__, ' '.join(info))
+
+    def _unregister(self):
+        if self._wait_handle is None:
+            return
         try:
             _overlapped.UnregisterWait(self._wait_handle)
         except OSError as e:
             if e.winerror != _overlapped.ERROR_IO_PENDING:
                 raise
+            # ERROR_IO_PENDING is not an error, the wait was unregistered
+        self._wait_handle = None
+
+    def cancel(self):
+        self._unregister()
+        return super().cancel()
 
 
 class PipeServer(object):
@@ -208,6 +251,11 @@ class IocpProactor:
         self._registered = weakref.WeakSet()
         self._stopped_serving = weakref.WeakSet()
 
+    def __repr__(self):
+        return ('<%s overlapped#=%s result#=%s>'
+                % (self.__class__.__name__, len(self._cache),
+                   len(self._results)))
+
     def set_loop(self, loop):
         self._loop = loop
 
@@ -350,23 +398,19 @@ class IocpProactor:
         ov = _overlapped.Overlapped(NULL)
         wh = _overlapped.RegisterWaitWithQueue(
             handle, self._iocp, ov.address, ms)
-        f = _WaitHandleFuture(wh, loop=self._loop)
+        f = _WaitHandleFuture(handle, wh, loop=self._loop)
 
         def finish_wait_for_handle(trans, key, ov):
-            if not f.cancelled():
-                try:
-                    _overlapped.UnregisterWait(wh)
-                except OSError as e:
-                    if e.winerror != _overlapped.ERROR_IO_PENDING:
-                        raise
             # Note that this second wait means that we should only use
             # this with handles types where a successful wait has no
             # effect.  So events or processes are all right, but locks
             # or semaphores are not.  Also note if the handle is
             # signalled and then quickly reset, then we may return
             # False even though we have not timed out.
-            return (_winapi.WaitForSingleObject(handle, 0) ==
-                    _winapi.WAIT_OBJECT_0)
+            try:
+                return f._poll()
+            finally:
+                f._unregister()
 
         self._cache[ov.address] = (f, ov, None, finish_wait_for_handle)
         return f
@@ -455,7 +499,7 @@ class IocpProactor:
 
     def close(self):
         # Cancel remaining registered operations.
-        for address, (f, ov, obj, callback) in list(self._cache.items()):
+        for address, (fut, ov, obj, callback) in list(self._cache.items()):
             if obj is None:
                 # The operation was started with connect_pipe() which
                 # queues a task to Windows' thread pool.  This cannot
@@ -463,9 +507,17 @@ class IocpProactor:
                 del self._cache[address]
             else:
                 try:
-                    ov.cancel()
-                except OSError:
-                    pass
+                    fut.cancel()
+                except OSError as exc:
+                    if self._loop is not None:
+                        context = {
+                            'message': 'Cancelling a future failed',
+                            'exception': exc,
+                            'future': fut,
+                        }
+                        if fut._source_traceback:
+                            context['source_traceback'] = fut._source_traceback
+                        self._loop.call_exception_handler(context)
 
         while self._cache:
             if not self._poll(1):
@@ -476,6 +528,9 @@ class IocpProactor:
             _winapi.CloseHandle(self._iocp)
             self._iocp = None
 
+    def __del__(self):
+        self.close()
+
 
 class _WindowsSubprocessTransport(base_subprocess.BaseSubprocessTransport):
 

Lib/ctypes/test/test_python_api.py

@@ -42,9 +42,9 @@ class PythonAPITestCase(unittest.TestCase):
     # This test is unreliable, because it is possible that code in
     # unittest changes the refcount of the '42' integer.  So, it
     # is disabled by default.
-    @requires("refcount")
     @support.refcount_test
     def test_PyLong_Long(self):
+        requires("refcount")
         ref42 = grc(42)
         pythonapi.PyLong_FromLong.restype = py_object
         self.assertEqual(pythonapi.PyLong_FromLong(42), 42)

Lib/ctypes/test/test_win32.py

@@ -38,8 +38,11 @@ class WindowsTestCase(unittest.TestCase):
 
 @unittest.skipUnless(sys.platform == "win32", 'Windows-specific test')
 class FunctionCallTestCase(unittest.TestCase):
-    @requires("SEH")
+    @unittest.skipUnless('MSC' in sys.version, "SEH only supported by MSC")
+    @unittest.skipIf(sys.executable.endswith('_d.exe'),
+                     "SEH not enabled in debug builds")
     def test_SEH(self):
+        requires("SEH")
         # Call functions with invalid arguments, and make sure
         # that access violations are trapped and raise an
         # exception.

Lib/plistlib.py

@@ -984,18 +984,16 @@ def load(fp, *, fmt=None, use_builtin_types=True, dict_type=dict):
         fp.seek(0)
         for info in _FORMATS.values():
             if info['detect'](header):
-                p = info['parser'](
-                    use_builtin_types=use_builtin_types,
-                    dict_type=dict_type,
-                )
+                P = info['parser']
                 break
 
         else:
             raise InvalidFileException()
 
     else:
-        p = _FORMATS[fmt]['parser'](use_builtin_types=use_builtin_types)
+        P = _FORMATS[fmt]['parser']
 
+    p = P(use_builtin_types=use_builtin_types, dict_type=dict_type)
     return p.parse(fp)
 
 

Lib/test/support/__init__.py

@@ -85,7 +85,7 @@ __all__ = [
     "skip_unless_symlink", "requires_gzip", "requires_bz2", "requires_lzma",
     "bigmemtest", "bigaddrspacetest", "cpython_only", "get_attribute",
     "requires_IEEE_754", "skip_unless_xattr", "requires_zlib",
-    "anticipate_failure",
+    "anticipate_failure", "load_package_tests",
     # sys
     "is_jython", "check_impl_detail",
     # network
@@ -188,6 +188,25 @@ def anticipate_failure(condition):
         return unittest.expectedFailure
     return lambda f: f
 
+def load_package_tests(pkg_dir, loader, standard_tests, pattern):
+    """Generic load_tests implementation for simple test packages.
+
+    Most packages can implement load_tests using this function as follows:
+
+       def load_tests(*args):
+           return load_package_tests(os.path.dirname(__file__), *args)
+    """
+    if pattern is None:
+        pattern = "test*"
+    top_dir = os.path.dirname(              # Lib
+                  os.path.dirname(              # test
+                      os.path.dirname(__file__)))   # support
+    package_tests = loader.discover(start_dir=pkg_dir,
+                                    top_level_dir=top_dir,
+                                    pattern=pattern)
+    standard_tests.addTests(package_tests)
+    return standard_tests
+
 
 def import_fresh_module(name, fresh=(), blocked=(), deprecated=False):
     """Import and return a module, deliberately bypassing sys.modules.

Lib/test/test_asynchat.py

@@ -7,10 +7,12 @@ thread = support.import_module('_thread')
 
 import asynchat
 import asyncore
+import errno
 import socket
 import sys
 import time
 import unittest
+import unittest.mock
 try:
     import threading
 except ImportError:
@@ -273,6 +275,21 @@ class TestAsynchat_WithPoll(TestAsynchat):
     usepoll = True
 
 
+class TestAsynchatMocked(unittest.TestCase):
+    def test_blockingioerror(self):
+        # Issue #16133: handle_read() must ignore BlockingIOError
+        sock = unittest.mock.Mock()
+        sock.recv.side_effect = BlockingIOError(errno.EAGAIN)
+
+        dispatcher = asynchat.async_chat()
+        dispatcher.set_socket(sock)
+        self.addCleanup(dispatcher.del_channel)
+
+        with unittest.mock.patch.object(dispatcher, 'handle_error') as error:
+            dispatcher.handle_read()
+        self.assertFalse(error.called)
+
+
 class TestHelperFunctions(unittest.TestCase):
     def test_find_prefix_at_end(self):
         self.assertEqual(asynchat.find_prefix_at_end("qwerty\r", "\r\n"), 1)

Lib/test/test_asyncio/__init__.py

 import os
-import sys
-import unittest
-from test.support import run_unittest, import_module
+from test.support import load_package_tests, import_module
 
 # Skip tests if we don't have threading.
 import_module('threading')
 # Skip tests if we don't have concurrent.futures.
 import_module('concurrent.futures')
 
-
-def suite():
-    tests = unittest.TestSuite()
-    loader = unittest.TestLoader()
-    for fn in os.listdir(os.path.dirname(__file__)):
-        if fn.startswith("test") and fn.endswith(".py"):
-            mod_name = 'test.test_asyncio.' + fn[:-3]
-            try:
-                __import__(mod_name)
-            except unittest.SkipTest:
-                pass
-            else:
-                mod = sys.modules[mod_name]
-                tests.addTests(loader.loadTestsFromModule(mod))
-    return tests
-
-
-def test_main():
-    run_unittest(suite())
+def load_tests(*args):
+    return load_package_tests(os.path.dirname(__file__), *args)

Lib/test/test_asyncio/__main__.py

-from . import test_main
+from . import load_tests
+import unittest
 
-
-if __name__ == '__main__':
-    test_main()
+unittest.main()

Lib/test/test_asyncio/test_selector_events.py

@@ -672,6 +672,8 @@ class SelectorTransportTests(test_utils.TestCase):
     def test_connection_lost(self):
         exc = OSError()
         tr = _SelectorTransport(self.loop, self.sock, self.protocol, None)
+        self.assertIsNotNone(tr._protocol)
+        self.assertIsNotNone(tr._loop)
         tr._call_connection_lost(exc)
 
         self.protocol.connection_lost.assert_called_with(exc)
@@ -679,8 +681,6 @@ class SelectorTransportTests(test_utils.TestCase):
         self.assertIsNone(tr._sock)
 
         self.assertIsNone(tr._protocol)
-        self.assertEqual(2, sys.getrefcount(self.protocol),
-                         pprint.pformat(gc.get_referrers(self.protocol)))
         self.assertIsNone(tr._loop)
 
 

Lib/test/test_asyncio/test_subprocess.py

@@ -42,7 +42,7 @@ class SubprocessMixin:
             return (exitcode, data)
 
         task = run(b'some data')
-        task = asyncio.wait_for(task, 10.0, loop=self.loop)
+        task = asyncio.wait_for(task, 60.0, loop=self.loop)
         exitcode, stdout = self.loop.run_until_complete(task)
         self.assertEqual(exitcode, 0)
         self.assertEqual(stdout, b'some data')
@@ -61,7 +61,7 @@ class SubprocessMixin:
             return proc.returncode, stdout
 
         task = run(b'some data')
-        task = asyncio.wait_for(task, 10.0, loop=self.loop)
+        task = asyncio.wait_for(task, 60.0, loop=self.loop)
         exitcode, stdout = self.loop.run_until_complete(task)
         self.assertEqual(exitcode, 0)
         self.assertEqual(stdout, b'some data')

Lib/test/test_asyncio/test_unix_events.py

@@ -435,6 +435,8 @@ class UnixReadPipeTransportTests(test_utils.TestCase):
     def test__call_connection_lost(self):
         tr = unix_events._UnixReadPipeTransport(
             self.loop, self.pipe, self.protocol)
+        self.assertIsNotNone(tr._protocol)
+        self.assertIsNotNone(tr._loop)
 
         err = None
         tr._call_connection_lost(err)
@@ -442,13 +444,13 @@ class UnixReadPipeTransportTests(test_utils.TestCase):
         self.pipe.close.assert_called_with()
 
         self.assertIsNone(tr._protocol)
-        self.assertEqual(2, sys.getrefcount(self.protocol),
-                         pprint.pformat(gc.get_referrers(self.protocol)))
         self.assertIsNone(tr._loop)
 
     def test__call_connection_lost_with_err(self):
         tr = unix_events._UnixReadPipeTransport(
             self.loop, self.pipe, self.protocol)
+        self.assertIsNotNone(tr._protocol)
+        self.assertIsNotNone(tr._loop)
 
         err = OSError()
         tr._call_connection_lost(err)
@@ -456,9 +458,6 @@ class UnixReadPipeTransportTests(test_utils.TestCase):
         self.pipe.close.assert_called_with()
 
         self.assertIsNone(tr._protocol)
-
-        self.assertEqual(2, sys.getrefcount(self.protocol),
-                         pprint.pformat(gc.get_referrers(self.protocol)))
         self.assertIsNone(tr._loop)
 
 
@@ -717,6 +716,8 @@ class UnixWritePipeTransportTests(test_utils.TestCase):
     def test__call_connection_lost(self):
         tr = unix_events._UnixWritePipeTransport(
             self.loop, self.pipe, self.protocol)
+        self.assertIsNotNone(tr._protocol)
+        self.assertIsNotNone(tr._loop)
 
         err = None
         tr._call_connection_lost(err)
@@ -724,13 +725,13 @@ class UnixWritePipeTransportTests(test_utils.TestCase):
         self.pipe.close.assert_called_with()
 
         self.assertIsNone(tr._protocol)
-        self.assertEqual(2, sys.getrefcount(self.protocol),
-                         pprint.pformat(gc.get_referrers(self.protocol)))
         self.assertIsNone(tr._loop)
 
     def test__call_connection_lost_with_err(self):
         tr = unix_events._UnixWritePipeTransport(
             self.loop, self.pipe, self.protocol)
+        self.assertIsNotNone(tr._protocol)
+        self.assertIsNotNone(tr._loop)
 
         err = OSError()
         tr._call_connection_lost(err)
@@ -738,8 +739,6 @@ class UnixWritePipeTransportTests(test_utils.TestCase):
         self.pipe.close.assert_called_with()
 
         self.assertIsNone(tr._protocol)
-        self.assertEqual(2, sys.getrefcount(self.protocol),
-                         pprint.pformat(gc.get_referrers(self.protocol)))
         self.assertIsNone(tr._loop)
 
     def test_close(self):

Lib/test/test_asyncio/test_windows_events.py

@@ -94,38 +94,48 @@ class ProactorTests(test_utils.TestCase):
         event = _overlapped.CreateEvent(None, True, False, None)
         self.addCleanup(_winapi.CloseHandle, event)
 
-        # Wait for unset event with 0.2s timeout;
+        # Wait for unset event with 0.5s timeout;
         # result should be False at timeout
-        f = self.loop._proactor.wait_for_handle(event, 0.2)
+        fut = self.loop._proactor.wait_for_handle(event, 0.5)
         start = self.loop.time()
-        self.loop.run_until_complete(f)
+        self.loop.run_until_complete(fut)
         elapsed = self.loop.time() - start
-        self.assertFalse(f.result())
-        self.assertTrue(0.18 < elapsed < 0.9, elapsed)
+        self.assertFalse(fut.result())
+        self.assertTrue(0.48 < elapsed < 0.9, elapsed)
 
         _overlapped.SetEvent(event)
 
         # Wait for for set event;
         # result should be True immediately
-        f = self.loop._proactor.wait_for_handle(event, 10)
+        fut = self.loop._proactor.wait_for_handle(event, 10)
         start = self.loop.time()
-        self.loop.run_until_complete(f)
+        self.loop.run_until_complete(fut)
         elapsed = self.loop.time() - start
-        self.assertTrue(f.result())
-        self.assertTrue(0 <= elapsed < 0.1, elapsed)
+        self.assertTrue(fut.result())
+        self.assertTrue(0 <= elapsed < 0.3, elapsed)
 
-        _overlapped.ResetEvent(event)
+        # Tulip issue #195: cancelling a done _WaitHandleFuture must not crash
+        fut.cancel()
+
+    def test_wait_for_handle_cancel(self):
+        event = _overlapped.CreateEvent(None, True, False, None)
+        self.addCleanup(_winapi.CloseHandle, event)
 
         # Wait for unset event with a cancelled future;
         # CancelledError should be raised immediately
-        f = self.loop._proactor.wait_for_handle(event, 10)
-        f.cancel()
+        fut = self.loop._proactor.wait_for_handle(event, 10)
+        fut.cancel()
         start = self.loop.time()
         with self.assertRaises(asyncio.CancelledError):
-            self.loop.run_until_complete(f)
+            self.loop.run_until_complete(fut)
         elapsed = self.loop.time() - start
         self.assertTrue(0 <= elapsed < 0.1, elapsed)
 
+        # Tulip issue #195: cancelling a _WaitHandleFuture twice must not crash
+        fut = self.loop._proactor.wait_for_handle(event)
+        fut.cancel()
+        fut.cancel()
+
 
 if __name__ == '__main__':
     unittest.main()

Lib/test/test_email/__init__.py

 import os
 import sys
 import unittest
-import test.support
 import collections
 import email
 from email.message import Message
 from email._policybase import compat32
+from test.support import load_package_tests
 from test.test_email import __file__ as landmark
 
-# Run all tests in package for '-m unittest test.test_email'
-def load_tests(loader, standard_tests, pattern):
-    this_dir = os.path.dirname(__file__)
-    if pattern is None:
-        pattern = "test*"
-    package_tests = loader.discover(start_dir=this_dir, pattern=pattern)
-    standard_tests.addTests(package_tests)
-    return standard_tests
-
-
-# used by regrtest and __main__.
-def test_main():
-    here = os.path.dirname(__file__)
-    # Unittest mucks with the path, so we have to save and restore
-    # it to keep regrtest happy.
-    savepath = sys.path[:]
-    test.support._run_suite(unittest.defaultTestLoader.discover(here))
-    sys.path[:] = savepath
+# Load all tests in package
+def load_tests(*args):
+    return load_package_tests(os.path.dirname(__file__), *args)
 
 
 # helper code used by a number of test modules.

Lib/test/test_email/__main__.py

-from test.test_email import test_main
+from test.test_email import load_tests
+import unittest
 
-test_main()
+unittest.main()

Lib/test/test_gettext.py

@@ -77,7 +77,7 @@ class GettextBaseTest(unittest.TestCase):
     def tearDown(self):
         self.env.__exit__()
         del self.env
-        shutil.rmtree(os.path.split(LOCALEDIR)[0])
+        support.rmtree(os.path.split(LOCALEDIR)[0])
 
 
 class GettextTestCase1(GettextBaseTest):

Lib/test/test_importlib/__init__.py

 import os
-import sys
-from test import support
-import unittest
+from test.support import load_package_tests
 
-def test_suite(package=__package__, directory=os.path.dirname(__file__)):
-    suite = unittest.TestSuite()
-    for name in os.listdir(directory):
-        if name.startswith(('.', '__')):
-            continue
-        path = os.path.join(directory, name)
-        if (os.path.isfile(path) and name.startswith('test_') and
-                name.endswith('.py')):
-            submodule_name = os.path.splitext(name)[0]
-            module_name = "{0}.{1}".format(package, submodule_name)
-            __import__(module_name, level=0)
-            module_tests = unittest.findTestCases(sys.modules[module_name])
-            suite.addTest(module_tests)
-        elif os.path.isdir(path):
-            package_name = "{0}.{1}".format(package, name)
-            __import__(package_name, level=0)
-            package_tests = getattr(sys.modules[package_name], 'test_suite')()
-            suite.addTest(package_tests)
-        else:
-            continue
-    return suite
-
-
-def test_main():
-    start_dir = os.path.dirname(__file__)
-    top_dir = os.path.dirname(os.path.dirname(start_dir))
-    test_loader = unittest.TestLoader()
-    support.run_unittest(test_loader.discover(start_dir, top_level_dir=top_dir))
+def load_tests(*args):
+    return load_package_tests(os.path.dirname(__file__), *args)

Lib/test/test_importlib/__main__.py

-"""Run importlib's test suite.
+from . import load_tests
+import unittest
 
-Specifying the ``--builtin`` flag will run tests, where applicable, with
-builtins.__import__ instead of importlib.__import__.
-
-"""
-if __name__ == '__main__':
-    from . import test_main
-    test_main()
+unittest.main()

Lib/test/test_importlib/builtin/__init__.py

-from .. import test_suite
 import os
+from test.support import load_package_tests
 
-
-def test_suite():
-    directory = os.path.dirname(__file__)
-    return test_suite('importlib.test.builtin', directory)
-
-
-if __name__ == '__main__':
-    from test.support import run_unittest
-    run_unittest(test_suite())
+def load_tests(*args):
+    return load_package_tests(os.path.dirname(__file__), *args)

Lib/test/test_importlib/builtin/__main__.py

+from . import load_tests
+import unittest
+
+unittest.main()

Lib/test/test_importlib/extension/__init__.py

-from .. import test_suite
-import os.path
-import unittest
+import os
+from test.support import load_package_tests
 
-
-def test_suite():
-    directory = os.path.dirname(__file__)
-    return test_suite('importlib.test.extension', directory)
-
-
-if __name__ == '__main__':
-    from test.support import run_unittest
-    run_unittest(test_suite())
+def load_tests(*args):
+    return load_package_tests(os.path.dirname(__file__), *args)

Lib/test/test_importlib/extension/__main__.py

+from . import load_tests
+import unittest
+
+unittest.main()

Lib/test/test_importlib/frozen/__init__.py

-from .. import test_suite
-import os.path
-import unittest
+import os
+from test.support import load_package_tests
 
-
-def test_suite():
-    directory = os.path.dirname(__file__)
-    return test_suite('importlib.test.frozen', directory)
-
-
-if __name__ == '__main__':
-    from test.support import run_unittest
-    run_unittest(test_suite())
+def load_tests(*args):
+    return load_package_tests(os.path.dirname(__file__), *args)

Lib/test/test_importlib/frozen/__main__.py

+from . import load_tests
+import unittest
+
+unittest.main()

Lib/test/test_importlib/import_/__init__.py

-from .. import test_suite
-import os.path
-import unittest
+import os
+from test.support import load_package_tests
 
-
-def test_suite():
-    directory = os.path.dirname(__file__)
-    return test_suite('importlib.test.import_', directory)
-
-
-if __name__ == '__main__':
-    from test.support import run_unittest
-    run_unittest(test_suite())
+def load_tests(*args):
+    return load_package_tests(os.path.dirname(__file__), *args)

Lib/test/test_importlib/import_/__main__.py

+from . import load_tests
+import unittest
+
+unittest.main()

Lib/test/test_importlib/source/__init__.py

-from .. import test_suite
-import os.path
-import unittest
+import os
+from test.support import load_package_tests
 
-
-def test_suite():
-    directory = os.path.dirname(__file__)
-    return test.test_suite('importlib.test.source', directory)
-
-
-if __name__ == '__main__':
-    from test.support import run_unittest
-    run_unittest(test_suite())
+def load_tests(*args):
+    return load_package_tests(os.path.dirname(__file__), *args)

Lib/test/test_importlib/source/__main__.py

+from . import load_tests
+import unittest
+
+unittest.main()

Lib/test/test_json/__init__.py

@@ -42,23 +42,12 @@ class TestCTest(CTest):
                          '_json')
 
 
-here = os.path.dirname(__file__)
-
-def load_tests(*args):
-    suite = additional_tests()
-    loader = unittest.TestLoader()
-    for fn in os.listdir(here):
-        if fn.startswith("test") and fn.endswith(".py"):
-            modname = "test.test_json." + fn[:-3]
-            __import__(modname)
-            module = sys.modules[modname]
-            suite.addTests(loader.loadTestsFromModule(module))
-    return suite
-
-def additional_tests():
+def load_tests(loader, _, pattern):
     suite = unittest.TestSuite()
     for mod in (json, json.encoder, json.decoder):
         suite.addTest(doctest.DocTestSuite(mod))
     suite.addTest(TestPyTest('test_pyjson'))
     suite.addTest(TestCTest('test_cjson'))
-    return suite
+
+    pkg_dir = os.path.dirname(__file__)
+    return support.load_package_tests(pkg_dir, loader, suite, pattern)

Lib/test/test_plistlib.py

@@ -207,6 +207,9 @@ class TestPlistlib(unittest.TestCase):
         for fmt in ALL_FORMATS:
             with self.subTest(fmt=fmt):
                 pl = self._create(fmt=fmt)
+                pl2 = plistlib.loads(TESTDATA[fmt], fmt=fmt)
+                self.assertEqual(dict(pl), dict(pl2),
+                    "generated data was not identical to Apple's output")
                 pl2 = plistlib.loads(TESTDATA[fmt])
                 self.assertEqual(dict(pl), dict(pl2),
                     "generated data was not identical to Apple's output")
@@ -217,6 +220,8 @@ class TestPlistlib(unittest.TestCase):
                 b = BytesIO()
                 pl = self._create(fmt=fmt)
                 plistlib.dump(pl, b, fmt=fmt)
+                pl2 = plistlib.load(BytesIO(b.getvalue()), fmt=fmt)
+                self.assertEqual(dict(pl), dict(pl2))
                 pl2 = plistlib.load(BytesIO(b.getvalue()))
                 self.assertEqual(dict(pl), dict(pl2))
 

Lib/test/test_readline.py

 """
 Very minimal unittests for parts of the readline module.
-
-These tests were added to check that the libedit emulation on OSX and
-the "real" readline have the same interface for history manipulation. That's
-why the tests cover only a small subset of the interface.
 """
+import os
 import unittest
 from test.support import run_unittest, import_module
+from test.script_helper import assert_python_ok
 
 # Skip tests if there is no readline module
 readline = import_module('readline')
 
 class TestHistoryManipulation (unittest.TestCase):
+    """
+    These tests were added to check that the libedit emulation on OSX and the
+    "real" readline have the same interface for history manipulation. That's
+    why the tests cover only a small subset of the interface.
+    """
 
     @unittest.skipIf(not hasattr(readline, 'clear_history'),
                      "The history update test cannot be run because the "
@@ -40,8 +43,18 @@ class TestHistoryManipulation (unittest.TestCase):
         self.assertEqual(readline.get_current_history_length(), 1)
 
 
+class TestReadline(unittest.TestCase):
+    def test_init(self):
+        # Issue #19884: Ensure that the ANSI sequence "\033[1034h" is not
+        # written into stdout when the readline module is imported and stdout
+        # is redirected to a pipe.
+        rc, stdout, stderr = assert_python_ok('-c', 'import readline',
+                                              TERM='xterm-256color')
+        self.assertEqual(stdout, b'')
+
+
 def test_main():
-    run_unittest(TestHistoryManipulation)
+    run_unittest(TestHistoryManipulation, TestReadline)
 
 if __name__ == "__main__":
     test_main()

Lib/test/test_tools/__init__.py

@@ -21,11 +21,5 @@ def import_tool(toolname):
     with support.DirsOnSysPath(scriptsdir):
         return importlib.import_module(toolname)
 
-def load_tests(loader, standard_tests, pattern):
-    this_dir = os.path.dirname(__file__)
-    if pattern is None:
-        pattern = "test*"
-    with support.DirsOnSysPath():
-        package_tests = loader.discover(start_dir=this_dir, pattern=pattern)
-    standard_tests.addTests(package_tests)
-    return standard_tests
+def load_tests(*args):
+    return support.load_package_tests(os.path.dirname(__file__), *args)

Lib/tkinter/__init__.py

@@ -421,7 +421,10 @@ class Misc:
               + _flatten(args) + _flatten(list(kw.items())))
     def tk_menuBar(self, *args):
         """Do not use. Needed in Tk 3.6 and earlier."""
-        pass # obsolete since Tk 4.0
+        # obsolete since Tk 4.0
+        import warnings
+        warnings.warn('tk_menuBar() does nothing and will be removed in 3.6',
+                      DeprecationWarning, stacklevel=2)
     def wait_variable(self, name='PY_VAR'):
         """Wait until the variable is modified.
 
@@ -2674,7 +2677,11 @@ class Menu(Widget):
         selectcolor, takefocus, tearoff, tearoffcommand, title, type."""
         Widget.__init__(self, master, 'menu', cnf, kw)
     def tk_bindForTraversal(self):
-        pass # obsolete since Tk 4.0
+        # obsolete since Tk 4.0
+        import warnings
+        warnings.warn('tk_bindForTraversal() does nothing and '
+                      'will be removed in 3.6',
+                      DeprecationWarning, stacklevel=2)
     def tk_mbPost(self):
         self.tk.call('tk_mbPost', self._w)
     def tk_mbUnpost(self):

Lib/tkinter/test/test_tkinter/test_widgets.py

@@ -916,6 +916,24 @@ class ScrollbarTest(AbstractWidgetTest, unittest.TestCase):
         self.checkEnumParam(widget, 'orient', 'vertical', 'horizontal',
                 errmsg='bad orientation "{}": must be vertical or horizontal')
 
+    def test_activate(self):
+        sb = self.create()
+        for e in ('arrow1', 'slider', 'arrow2'):
+            sb.activate(e)
+        sb.activate('')
+        self.assertRaises(TypeError, sb.activate)
+        self.assertRaises(TypeError, sb.activate, 'arrow1', 'arrow2')
+
+    def test_set(self):
+        sb = self.create()
+        sb.set(0.2, 0.4)
+        self.assertEqual(sb.get(), (0.2, 0.4))
+        self.assertRaises(TclError, sb.set, 'abc', 'def')
+        self.assertRaises(TclError, sb.set, 0.6, 'def')
+        self.assertRaises(TclError, sb.set, 0.6, None)
+        self.assertRaises(TclError, sb.set, 0.6)
+        self.assertRaises(TclError, sb.set, 0.6, 0.7, 0.8)
+
 
 @add_standard_options(StandardOptionsTests)
 class PanedWindowTest(AbstractWidgetTest, unittest.TestCase):

Lib/turtledemo/__main__.py

@@ -27,99 +27,102 @@ def getExampleEntries():
     return [entry[:-3] for entry in os.listdir(demo_dir) if
             entry.endswith(".py") and entry[0] != '_']
 
-def showDemoHelp():
-    view_file(demo.root, "Help on turtleDemo",
-              os.path.join(demo_dir, "demohelp.txt"))
-
-def showAboutDemo():
-    view_file(demo.root, "About turtleDemo",
-              os.path.join(demo_dir, "about_turtledemo.txt"))
-
-def showAboutTurtle():
-    view_file(demo.root, "About the new turtle module.",
-              os.path.join(demo_dir, "about_turtle.txt"))
+help_entries = (  # (help_label,  help_file)
+    ('Turtledemo help', "demohelp.txt"),
+    ('About turtledemo', "about_turtledemo.txt"),
+    ('About turtle module', "about_turtle.txt"),
+    )
 
 class DemoWindow(object):
 
-    def __init__(self, filename=None):   #, root=None):
+    def __init__(self, filename=None):
         self.root = root = turtle._root = Tk()
+        root.title('Python turtle-graphics examples')
         root.wm_protocol("WM_DELETE_WINDOW", self._destroy)
 
-        #################
-        self.mBar = Frame(root, relief=RAISED, borderwidth=2)
-        self.mBar.pack(fill=X)
+        root.grid_rowconfigure(1, weight=1)
+        root.grid_columnconfigure(0, weight=1)
+        root.grid_columnconfigure(1, minsize=90, weight=1)
+        root.grid_columnconfigure(2, minsize=90, weight=1)
+        root.grid_columnconfigure(3, minsize=90, weight=1)
 
+        self.mBar = Frame(root, relief=RAISED, borderwidth=2)
         self.ExamplesBtn = self.makeLoadDemoMenu()
         self.OptionsBtn = self.makeHelpMenu()
-        self.mBar.tk_menuBar(self.ExamplesBtn, self.OptionsBtn) #, QuitBtn)
+        self.mBar.grid(row=0, columnspan=4, sticky='news')
+
+        pane = PanedWindow(orient=HORIZONTAL, sashwidth=5,
+                           sashrelief=SOLID, bg='#ddd')
+        pane.add(self.makeTextFrame(pane))
+        pane.add(self.makeGraphFrame(pane))
+        pane.grid(row=1, columnspan=4, sticky='news')
+
+        self.output_lbl = Label(root, height= 1, text=" --- ", bg="#ddf",
+                                font=("Arial", 16, 'normal'), borderwidth=2,
+                                relief=RIDGE)
+        self.start_btn = Button(root, text=" START ", font=btnfont,
+                                fg="white", disabledforeground = "#fed",
+                                command=self.startDemo)
+        self.stop_btn = Button(root, text=" STOP ", font=btnfont,
+                               fg="white", disabledforeground = "#fed",
+                               command=self.stopIt)
+        self.clear_btn = Button(root, text=" CLEAR ", font=btnfont,
+                                fg="white", disabledforeground="#fed",
+                                command = self.clearCanvas)
+        self.output_lbl.grid(row=2, column=0, sticky='news', padx=(0,5))
+        self.start_btn.grid(row=2, column=1, sticky='ew')
+        self.stop_btn.grid(row=2, column=2, sticky='ew')
+        self.clear_btn.grid(row=2, column=3, sticky='ew')
+
+        Percolator(self.text).insertfilter(ColorDelegator())
+        self.dirty = False
+        self.exitflag = False
+        if filename:
+            self.loadfile(filename)
+        self.configGUI(NORMAL, DISABLED, DISABLED, DISABLED,
+                       "Choose example from menu", "black")
+        self.state = STARTUP
 
-        root.title('Python turtle-graphics examples')
-        #################
-        self.left_frame = left_frame = Frame(root)
-        self.text_frame = text_frame = Frame(left_frame)
-        self.vbar = vbar =Scrollbar(text_frame, name='vbar')
-        self.text = text = Text(text_frame,
-                                name='text', padx=5, wrap='none',
-                                width=45)
+
+    def onResize(self, event):
+        cwidth = self._canvas.winfo_width()
+        cheight = self._canvas.winfo_height()
+        self._canvas.xview_moveto(0.5*(self.canvwidth-cwidth)/self.canvwidth)
+        self._canvas.yview_moveto(0.5*(self.canvheight-cheight)/self.canvheight)
+
+    def makeTextFrame(self, root):
+        self.text_frame = text_frame = Frame(root)
+        self.text = text = Text(text_frame, name='text', padx=5,
+                                wrap='none', width=45)
+
+        self.vbar = vbar = Scrollbar(text_frame, name='vbar')
         vbar['command'] = text.yview
         vbar.pack(side=LEFT, fill=Y)
-        #####################
-        self.hbar = hbar =Scrollbar(text_frame, name='hbar', orient=HORIZONTAL)
+        self.hbar = hbar = Scrollbar(text_frame, name='hbar', orient=HORIZONTAL)
         hbar['command'] = text.xview
         hbar.pack(side=BOTTOM, fill=X)
-        #####################
+
+        text['font'] = txtfont
         text['yscrollcommand'] = vbar.set
-        text.config(font=txtfont)
-        text.config(xscrollcommand=hbar.set)
-        text.pack(side=LEFT, fill=Y, expand=1)
-        #####################
-        self.output_lbl = Label(left_frame, height= 1,text=" --- ", bg = "#ddf",
-                                font = ("Arial", 16, 'normal'))
-        self.output_lbl.pack(side=BOTTOM, expand=0, fill=X)
-        #####################
-        text_frame.pack(side=LEFT, fill=BOTH, expand=0)
-        left_frame.pack(side=LEFT, fill=BOTH, expand=0)
-        self.graph_frame = g_frame = Frame(root)
-
-        turtle._Screen._root = g_frame
-        turtle._Screen._canvas = turtle.ScrolledCanvas(g_frame, 800, 600, 1000, 800)
-        #xturtle.Screen._canvas.pack(expand=1, fill="both")
+        text['xscrollcommand'] = hbar.set
+        text.pack(side=LEFT, fill=BOTH, expand=1)
+        return text_frame
+
+    def makeGraphFrame(self, root):
+        turtle._Screen._root = root
+        self.canvwidth = 1000
+        self.canvheight = 800
+        turtle._Screen._canvas = self._canvas = canvas = turtle.ScrolledCanvas(
+                root, 800, 600, self.canvwidth, self.canvheight)
+        canvas.adjustScrolls()
+        canvas._rootwindow.bind('<Configure>', self.onResize)
+        canvas._canvas['borderwidth'] = 0
+
         self.screen = _s_ = turtle.Screen()
-#####
         turtle.TurtleScreen.__init__(_s_, _s_._canvas)
-#####
         self.scanvas = _s_._canvas
-        #xturtle.RawTurtle.canvases = [self.scanvas]
         turtle.RawTurtle.screens = [_s_]
-
-        self.scanvas.pack(side=TOP, fill=BOTH, expand=1)
-
-        self.btn_frame = btn_frame = Frame(g_frame, height=100)
-        self.start_btn = Button(btn_frame, text=" START ", font=btnfont, fg = "white",
-                                disabledforeground = "#fed", command=self.startDemo)
-        self.start_btn.pack(side=LEFT, fill=X, expand=1)
-        self.stop_btn = Button(btn_frame, text=" STOP ",  font=btnfont, fg = "white",
-                                disabledforeground = "#fed", command = self.stopIt)
-        self.stop_btn.pack(side=LEFT, fill=X, expand=1)
-        self.clear_btn = Button(btn_frame, text=" CLEAR ",  font=btnfont, fg = "white",
-                                disabledforeground = "#fed", command = self.clearCanvas)
-        self.clear_btn.pack(side=LEFT, fill=X, expand=1)
-
-        self.btn_frame.pack(side=TOP, fill=BOTH, expand=0)
-        self.graph_frame.pack(side=TOP, fill=BOTH, expand=1)
-
-        Percolator(text).insertfilter(ColorDelegator())
-        self.dirty = False
-        self.exitflag = False
-        if filename:
-            self.loadfile(filename)
-        self.configGUI(NORMAL, DISABLED, DISABLED, DISABLED,
-                       "Choose example from menu", "black")
-        self.state = STARTUP
-
-    def _destroy(self):
-        self.root.destroy()
-        sys.exit()
+        return canvas
 
     def configGUI(self, menu, start, stop, clear, txt="", color="blue"):
         self.ExamplesBtn.config(state=menu)
@@ -145,9 +148,9 @@ class DemoWindow(object):
 
         self.output_lbl.config(text=txt, fg=color)
 
-
     def makeLoadDemoMenu(self):
-        CmdBtn = Menubutton(self.mBar, text='Examples', underline=0, font=menufont)
+        CmdBtn = Menubutton(self.mBar, text='Examples',
+                            underline=0, font=menufont)
         CmdBtn.pack(side=LEFT, padx="2m")
         CmdBtn.menu = Menu(CmdBtn)
 
@@ -167,12 +170,10 @@ class DemoWindow(object):
         CmdBtn.pack(side=LEFT, padx='2m')
         CmdBtn.menu = Menu(CmdBtn)
 
-        CmdBtn.menu.add_command(label='About turtle.py', font=menufont,
-                                command=showAboutTurtle)
-        CmdBtn.menu.add_command(label='turtleDemo - Help', font=menufont,
-                                command=showDemoHelp)
-        CmdBtn.menu.add_command(label='About turtleDemo', font=menufont,
-                                command=showAboutDemo)
+        for help_label, help_file in help_entries:
+            def show(help_label=help_label, help_file=help_file):
+                view_file(self.root, help_label, os.path.join(demo_dir, help_file))
+            CmdBtn.menu.add_command(label=help_label, font=menufont, command=show)
 
         CmdBtn['menu'] = CmdBtn.menu
         return CmdBtn
@@ -180,7 +181,6 @@ class DemoWindow(object):
     def refreshCanvas(self):
         if not self.dirty: return
         self.screen.clear()
-        #self.screen.mode("standard")
         self.dirty=False
 
     def loadfile(self, filename):
@@ -238,29 +238,16 @@ class DemoWindow(object):
             self.configGUI(NORMAL, NORMAL, DISABLED, DISABLED,
                            "STOPPED!", "red")
             turtle.TurtleScreen._RUNNING = False
-            #print "stopIT: exitflag = True"
         else:
             turtle.TurtleScreen._RUNNING = False
-            #print "stopIt: exitflag = False"
 
-if __name__ == '__main__':
+    def _destroy(self):
+        self.root.destroy()
+
+
+def main():
     demo = DemoWindow()
-    RUN = True
-    while RUN:
-        try:
-            #print("ENTERING mainloop")
-            demo.root.mainloop()
-        except AttributeError:
-            #print("AttributeError!- WAIT A MOMENT!")
-            time.sleep(0.3)
-            print("GOING ON ..")
-            demo.ckearCanvas()
-        except TypeError:
-            demo.screen._delete("all")
-            #print("CRASH!!!- WAIT A MOMENT!")
-            time.sleep(0.3)
-            #print("GOING ON ..")
-            demo.clearCanvas()
-        except:
-            print("BYE!")
-            RUN = False
+    demo.root.mainloop()
+
+if __name__ == '__main__':
+    main()

Misc/ACKS

@@ -662,6 +662,7 @@ Kurt B. Kaiser
 Tamito Kajiyama
 Jan Kaliszewski
 Peter van Kampen
+Jan Kanis
 Rafe Kaplan
 Jacob Kaplan-Moss
 Janne Karila
@@ -997,7 +998,6 @@ Mike Pall
 Todd R. Palmer
 Juan David Ibáñez Palomar
 Jan Palus
-Martin Panter
 Mathias Panzenböck
 M. Papillon
 Peter Parente

Misc/NEWS

@@ -27,8 +27,19 @@ Core and Builtins
 Library
 -------
 
+- Issue #16133: The asynchat.async_chat.handle_read() method now ignores
+  BlockingIOError exceptions.
+
+- Issue #19884: readline: Disable the meta modifier key if stdout is not
+  a terminal to not write the ANSI sequence "\033[1034h" into stdout. This
+  sequence is used on some terminal (ex: TERM=xterm-256color") to enable
+  support of 8 bit characters.
+
+- Issue #21888: plistlib's load() and loads() now work if the fmt parameter is
+  specified.
+
 - Issue #21044: tarfile.open() now handles fileobj with an integer 'name'
-  attribute.  Based on patch by Martin Panter.
+  attribute.  Based on patch by Antoine Pietri.
 
 - Issue #21867:  Prevent turtle crash due to invalid undo buffer size.
 
@@ -206,6 +217,10 @@ IDLE
 Tests
 -----
 
+- Issue #22002: Added ``load_package_tests`` function to test.support and used
+  it to implement/augment test discovery in test_asyncio, test_email,
+  test_importlib, test_json, and test_tools.
+
 - Issue #21976: Fix test_ssl to accept LibreSSL version strings.  Thanks
   to William Orr.
 

Modules/readline.c

@@ -1019,6 +1019,21 @@ setup_readline(readlinestate *mod_state)
 
     mod_state->begidx = PyLong_FromLong(0L);
     mod_state->endidx = PyLong_FromLong(0L);
+
+#ifndef __APPLE__
+    if (!isatty(STDOUT_FILENO)) {
+        /* Issue #19884: stdout is no a terminal. Disable meta modifier
+           keys to not write the ANSI sequence "\033[1034h" into stdout. On
+           terminals supporting 8 bit characters like TERM=xterm-256color
+           (which is now the default Fedora since Fedora 18), the meta key is
+           used to enable support of 8 bit characters (ANSI sequence
+           "\033[1034h").
+
+           With libedit, this call makes readline() crash. */
+        rl_variable_bind ("enable-meta-key", "off");
+    }
+#endif
+
     /* Initialize (allows .inputrc to override)
      *
      * XXX: A bug in the readline-2.2 library causes a memory leak

Objects/stringlib/README.txt

-bits shared by the stringobject and unicodeobject implementations (and
+bits shared by the bytesobject and unicodeobject implementations (and
 possibly other modules, in a not too distant future).
 
 the stuff in here is included into relevant places; see the individual