merge

Doc/c-api/typeobj.rst

@@ -192,7 +192,7 @@ type objects) *must* have the :attr:`ob_size` field.
    An optional pointer to the instance print function.
 
    The print function is only called when the instance is printed to a *real* file;
-   when it is printed to a pseudo-file (like a :class:`StringIO` instance), the
+   when it is printed to a pseudo-file (like a :class:`io.StringIO` instance), the
    instance's :c:member:`~PyTypeObject.tp_repr` or :c:member:`~PyTypeObject.tp_str` function is called to convert it to
    a string.  These are also called when the type's :c:member:`~PyTypeObject.tp_print` field is
    *NULL*.  A type should never implement :c:member:`~PyTypeObject.tp_print` in a way that produces

Doc/library/gzip.rst

@@ -62,7 +62,7 @@ The module defines the following items:
    value.
 
    The new class instance is based on *fileobj*, which can be a regular file, a
-   :class:`StringIO` object, or any other object which simulates a file.  It
+   :class:`io.BytesIO` object, or any other object which simulates a file.  It
    defaults to ``None``, in which case *filename* is opened to provide a file
    object.
 

Doc/library/mailbox.rst

@@ -674,8 +674,8 @@ Supported mailbox formats are Maildir, mbox, MH, Babyl, and MMDF.
 
       In Babyl mailboxes, the headers of a message are not stored contiguously
       with the body of the message. To generate a file-like representation, the
-      headers and body are copied together into a :class:`StringIO` instance
-      (from the :mod:`StringIO` module), which has an API identical to that of a
+      headers and body are copied together into a :class:`io.BytesIO` instance,
+      which has an API identical to that of a
       file. As a result, the file-like object is truly independent of the
       underlying mailbox but does not save memory compared to a string
       representation.

Doc/library/random.rst

@@ -45,9 +45,9 @@ from sources provided by the operating system.
 
 .. warning::
 
-   The generators of the :mod:`random` module should not be used for security
-   purposes. Use :func:`ssl.RAND_bytes` if you require a cryptographically
-   secure pseudorandom number generator.
+   The pseudo-random generators of this module should not be used for
+   security purposes.  Use :func:`os.urandom` or :class:`SystemRandom` if
+   you require a cryptographically secure pseudo-random number generator.
 
 
 Bookkeeping functions:

Doc/library/tempfile.rst

@@ -82,7 +82,7 @@ The module defines the following user-callable items:
    causes the file to roll over to an on-disk file regardless of its size.
 
    The returned object is a file-like object whose :attr:`_file` attribute
-   is either a :class:`BytesIO` or :class:`StringIO` object (depending on
+   is either a :class:`io.BytesIO` or :class:`io.StringIO` object (depending on
    whether binary or text *mode* was specified) or a true file
    object, depending on whether :func:`rollover` has been called.  This
    file-like object can be used in a :keyword:`with` statement, just like

Doc/library/unittest.mock.rst

@@ -1084,9 +1084,9 @@ you wanted a :class:`NonCallableMock` to be used:
       ...
     TypeError: 'NonCallableMock' object is not callable
 
-Another use case might be to replace an object with a `StringIO` instance:
+Another use case might be to replace an object with a `io.StringIO` instance:
 
-    >>> from StringIO import StringIO
+    >>> from io import StringIO
     >>> def foo():
     ...     print 'Something'
     ...

Doc/library/xml.dom.minidom.rst

@@ -55,7 +55,7 @@ instead:
 .. function:: parseString(string, parser=None)
 
    Return a :class:`Document` that represents the *string*. This method creates a
-   :class:`StringIO` object for the string and passes that on to :func:`parse`.
+   :class:`io.StringIO` object for the string and passes that on to :func:`parse`.
 
 Both functions return a :class:`Document` object representing the content of the
 document.

Lib/test/test_os.py

@@ -28,6 +28,11 @@ try:
     import threading
 except ImportError:
     threading = None
+try:
+    import resource
+except ImportError:
+    resource = None
+
 from test.script_helper import assert_python_ok
 
 with warnings.catch_warnings():
@@ -997,6 +1002,21 @@ class URandomTests(unittest.TestCase):
         data2 = self.get_urandom_subprocess(16)
         self.assertNotEqual(data1, data2)
 
+    @unittest.skipUnless(resource, "test requires the resource module")
+    def test_urandom_failure(self):
+        soft_limit, hard_limit = resource.getrlimit(resource.RLIMIT_NOFILE)
+        resource.setrlimit(resource.RLIMIT_NOFILE, (1, hard_limit))
+        try:
+            with self.assertRaises(OSError) as cm:
+                os.urandom(16)
+            self.assertEqual(cm.exception.errno, errno.EMFILE)
+        finally:
+            # We restore the old limit as soon as possible.  If doing it
+            # using addCleanup(), code running in between would fail
+            # creating any file descriptor.
+            resource.setrlimit(resource.RLIMIT_NOFILE, (soft_limit, hard_limit))
+
+
 @contextlib.contextmanager
 def _execvpe_mockup(defpath=None):
     """

Lib/test/test_shutil.py

@@ -726,6 +726,32 @@ class TestShutil(unittest.TestCase):
             shutil.rmtree(src_dir)
             shutil.rmtree(os.path.dirname(dst_dir))
 
+    def test_copytree_retains_permissions(self):
+        tmp_dir = tempfile.mkdtemp()
+        src_dir = os.path.join(tmp_dir, 'source')
+        os.mkdir(src_dir)
+        dst_dir = os.path.join(tmp_dir, 'destination')
+        self.addCleanup(shutil.rmtree, tmp_dir)
+
+        os.chmod(src_dir, 0o777)
+        write_file((src_dir, 'permissive.txt'), '123')
+        os.chmod(os.path.join(src_dir, 'permissive.txt'), 0o777)
+        write_file((src_dir, 'restrictive.txt'), '456')
+        os.chmod(os.path.join(src_dir, 'restrictive.txt'), 0o600)
+        restrictive_subdir = tempfile.mkdtemp(dir=src_dir)
+        os.chmod(restrictive_subdir, 0o600)
+
+        shutil.copytree(src_dir, dst_dir)
+        self.assertEquals(os.stat(src_dir).st_mode, os.stat(dst_dir).st_mode)
+        self.assertEquals(os.stat(os.path.join(src_dir, 'permissive.txt')).st_mode,
+                          os.stat(os.path.join(dst_dir, 'permissive.txt')).st_mode)
+        self.assertEquals(os.stat(os.path.join(src_dir, 'restrictive.txt')).st_mode,
+                          os.stat(os.path.join(dst_dir, 'restrictive.txt')).st_mode)
+        restrictive_subdir_dst = os.path.join(dst_dir,
+                                              os.path.split(restrictive_subdir)[1])
+        self.assertEquals(os.stat(restrictive_subdir).st_mode,
+                          os.stat(restrictive_subdir_dst).st_mode)
+
     @unittest.skipUnless(hasattr(os, 'link'), 'requires os.link')
     def test_dont_copy_file_onto_link_to_itself(self):
         # Temporarily disable test on Windows.

Lib/test/test_timeout.py

 """Unit tests for socket timeout feature."""
 
+import functools
 import unittest
 from test import support
 
@@ -11,6 +12,18 @@ import errno
 import socket
 
 
+@functools.lru_cache()
+def resolve_address(host, port):
+    """Resolve an (host, port) to an address.
+
+    We must perform name resolution before timeout tests, otherwise it will be
+    performed by connect().
+    """
+    with support.transient_internet(host):
+        return socket.getaddrinfo(host, port, socket.AF_INET,
+                                  socket.SOCK_STREAM)[0][4]
+
+
 class CreationTestCase(unittest.TestCase):
     """Test case for socket.gettimeout() and socket.settimeout()"""
 
@@ -132,7 +145,7 @@ class TCPTimeoutTestCase(TimeoutTestCase):
 
     def setUp(self):
         self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-        self.addr_remote = ('www.python.org.', 80)
+        self.addr_remote = resolve_address('www.python.org.', 80)
 
     def tearDown(self):
         self.sock.close()
@@ -142,7 +155,7 @@ class TCPTimeoutTestCase(TimeoutTestCase):
         # to a host that silently drops our packets.  We can't simulate this
         # from Python because it's a function of the underlying TCP/IP stack.
         # So, the following Snakebite host has been defined:
-        blackhole = ('blackhole.snakebite.net', 56666)
+        blackhole = resolve_address('blackhole.snakebite.net', 56666)
 
         # Blackhole has been configured to silently drop any incoming packets.
         # No RSTs (for TCP) or ICMP UNREACH (for UDP/ICMP) will be sent back
@@ -154,7 +167,7 @@ class TCPTimeoutTestCase(TimeoutTestCase):
         # to firewalling or general network configuration.  In order to improve
         # our confidence in testing the blackhole, a corresponding 'whitehole'
         # has also been set up using one port higher:
-        whitehole = ('whitehole.snakebite.net', 56667)
+        whitehole = resolve_address('whitehole.snakebite.net', 56667)
 
         # This address has been configured to immediately drop any incoming
         # packets as well, but it does it respectfully with regards to the

Misc/NEWS

@@ -72,6 +72,10 @@ Library
   strings for ``rfc822Name`` (email), ``dNSName`` (DNS) and
   ``uniformResourceIdentifier`` (URI).
 
+- Issue #18756: Improve error reporting in os.urandom() when the failure
+  is due to something else than /dev/urandom not existing (for example,
+  exhausting the file descriptor limit).
+
 - Issue #18405: Improve the entropy of crypt.mksalt().
 
 - Issue #18676: Change 'positive' to 'non-negative' in queue.py put and get
@@ -264,6 +268,9 @@ IDLE
 Tests
 -----
 
+- Issue #1666318: Add a test that shutil.copytree() retains directory
+  permissions.  Patch by Catherine Devlin.
+
 - Issue #18357: add tests for dictview set difference.
   Patch by Fraser Tweedale.
 

Python/random.c

@@ -165,8 +165,12 @@ dev_urandom_python(char *buffer, Py_ssize_t size)
     Py_END_ALLOW_THREADS
     if (fd < 0)
     {
+        if (errno == ENOENT || errno == ENXIO ||
+            errno == ENODEV || errno == EACCES)
             PyErr_SetString(PyExc_NotImplementedError,
                             "/dev/urandom (or equivalent) not found");
+        else
+            PyErr_SetFromErrno(PyExc_OSError);
         return -1;
     }
 

README

@@ -76,6 +76,13 @@ is downloadable in HTML, PDF, and reStructuredText formats; the latter version
 is primarily for documentation authors, translators, and people with special
 formatting requirements.
 
+If you would like to contribute to the development of Python, relevant
+documentation is available at:
+
+    http://docs.python.org/devguide/
+
+For information about building Python's documentation, refer to Doc/README.txt.
+
 
 Converting From Python 2.x to 3.x
 ---------------------------------