Issue #21682: Replace EditorWindow with mock to eliminate memory leaks.

Patch by Saimadhav Heblikar. (2 head merge)

Doc/library/os.rst

@@ -257,7 +257,7 @@ process and user.
 
    .. index:: single: user; id
 
-   Return the current process's user id.
+   Return the current process's real user id.
 
    Availability: Unix.
 

Doc/library/tarfile.rst

@@ -77,6 +77,10 @@ Some facts and figures:
    If *fileobj* is specified, it is used as an alternative to a file object opened
    for *name*. It is supposed to be at position 0.
 
+   For modes ``'w:gz'``, ``'r:gz'``, ``'w:bz2'``, ``'r:bz2'``, :func:`tarfile.open`
+   accepts the keyword argument *compresslevel* to specify the compression level of
+   the file.
+
    For special purposes, there is a second format for *mode*:
    ``'filemode|[compression]'``.  :func:`tarfile.open` will return a :class:`TarFile`
    object that processes its data as a stream of blocks.  No random seeking will

Doc/library/tokenize.rst

@@ -98,6 +98,24 @@ back the modified script.
 
    .. versionadded:: 2.5
 
+.. exception:: TokenError
+
+   Raised when either a docstring or expression that may be split over several
+   lines is not completed anywhere in the file, for example::
+
+      """Beginning of
+      docstring
+
+   or::
+
+      [1,
+       2,
+       3
+
+Note that unclosed single-quoted strings do not cause an error to be
+raised. They are tokenized as ``ERRORTOKEN``, followed by the tokenization of
+their contents.
+
 Example of a script re-writer that transforms float literals into Decimal
 objects::
 

Doc/whatsnew/2.7.rst

@@ -79,45 +79,91 @@ bug/patch item for each change.
 The Future for Python 2.x
 =========================
 
-Python 2.7 is intended to be the last major release in the 2.x series.
-The Python maintainers are planning to focus their future efforts on
-the Python 3.x series.
-
-This means that 2.7 will remain in place for a long time, running
-production systems that have not been ported to Python 3.x.
-Two consequences of the long-term significance of 2.7 are:
-
-* It's very likely the 2.7 release will have a longer period of
-  maintenance compared to earlier 2.x versions.  Python 2.7 will
-  continue to be maintained while the transition to 3.x continues, and
-  the developers are planning to support Python 2.7 with bug-fix
-  releases beyond the typical two years.
-
-* A policy decision was made to silence warnings only of interest to
-  developers.  :exc:`DeprecationWarning` and its
-  descendants are now ignored unless otherwise requested, preventing
-  users from seeing warnings triggered by an application.  This change
-  was also made in the branch that will become Python 3.2. (Discussed
-  on stdlib-sig and carried out in :issue:`7319`.)
-
-  In previous releases, :exc:`DeprecationWarning` messages were
-  enabled by default, providing Python developers with a clear
-  indication of where their code may break in a future major version
-  of Python.
-
-  However, there are increasingly many users of Python-based
-  applications who are not directly involved in the development of
-  those applications.  :exc:`DeprecationWarning` messages are
-  irrelevant to such users, making them worry about an application
-  that's actually working correctly and burdening application developers
-  with responding to these concerns.
-
-  You can re-enable display of :exc:`DeprecationWarning` messages by
-  running Python with the :option:`-Wdefault <-W>` (short form:
-  :option:`-Wd <-W>`) switch, or by setting the :envvar:`PYTHONWARNINGS`
-  environment variable to ``"default"`` (or ``"d"``) before running
-  Python.  Python code can also re-enable them
-  by calling ``warnings.simplefilter('default')``.
+Python 2.7 is the last major release in the 2.x series, as the Python
+maintainers have shifted the focus of their new feature development efforts
+to the Python 3.x series. This means that while Python 2 continues to
+receive bug fixes, and to be updated to build correctly on new hardware and
+versions of supported operated systems, there will be no new full feature
+releases for the language or standard library.
+
+However, while there is a large common subset between Python 2.7 and Python
+3, and many of the changes involved in migrating to that common subset, or
+directly to Python 3, can be safely automated, some other changes (notably
+those associated with Unicode handling) may require careful consideration,
+and preferably robust automated regression test suites, to migrate
+effectively.
+
+This means that Python 2.7 will remain in place for a long time, providing a
+stable and supported base platform for production systems that have not yet
+been ported to Python 3. The full expected lifecycle of the Python 2.7
+series is detailed in :pep:`373`.
+
+Some key consequences of the long-term significance of 2.7 are:
+
+* As noted above, the 2.7 release has a much longer period of maintenance
+  when compared to earlier 2.x versions. Python 2.7 is currently expected to
+  remain supported by the core development team (receiving security updates
+  and other bug fixes) until at least 2020 (10 years after its initial
+  release, compared to the more typical support period of 18-24 months).
+
+* As the Python 2.7 standard library ages, making effective use of the
+  Python Package Index (either directly or via a redistributor) becomes
+  more important for Python 2 users. In addition to a wide variety of third
+  party packages for various tasks, the available packages include backports
+  of new modules and features from the Python 3 standard library that are
+  compatible with Python 2, as well as various tools and libraries that can
+  make it easier to migrate to Python 3. The `Python Packaging User Guide
+  <https://packaging.python.org>`__ provides guidance on downloading and
+  installing software from the Python Package Index.
+
+* While the preferred approach to enhancing Python 2 is now the publication
+  of new packages on the Python Package Index, this approach doesn't
+  necessarily work in all cases, especially those related to network
+  security. In exceptional cases that cannot be handled adequately by
+  publishing new or updated packages on PyPI, the Python Enhancement
+  Proposal process may be used to make the case for adding new features
+  directly to the Python 2 standard library. Any such additions, and the
+  maintenance releases where they were added, will be noted in the
+  :ref:`py27-maintenance-enhancements` section below.
+
+For projects wishing to migrate from Python 2 to Python 3, or for library
+and framework developers wishing to support users on both Python 2 and
+Python 3, there are a variety of tools and guides available to help decide
+on a suitable approach and manage some of the technical details involved.
+The recommended starting point is the :ref:`pyporting-howto` HOWTO guide.
+
+
+Changes to the Handling of Deprecation Warnings
+===============================================
+
+For Python 2.7, a policy decision was made to silence warnings only of
+interest to developers by default.  :exc:`DeprecationWarning` and its
+descendants are now ignored unless otherwise requested, preventing
+users from seeing warnings triggered by an application.  This change
+was also made in the branch that became Python 3.2. (Discussed
+on stdlib-sig and carried out in :issue:`7319`.)
+
+In previous releases, :exc:`DeprecationWarning` messages were
+enabled by default, providing Python developers with a clear
+indication of where their code may break in a future major version
+of Python.
+
+However, there are increasingly many users of Python-based
+applications who are not directly involved in the development of
+those applications.  :exc:`DeprecationWarning` messages are
+irrelevant to such users, making them worry about an application
+that's actually working correctly and burdening application developers
+with responding to these concerns.
+
+You can re-enable display of :exc:`DeprecationWarning` messages by
+running Python with the :option:`-Wdefault <-W>` (short form:
+:option:`-Wd <-W>`) switch, or by setting the :envvar:`PYTHONWARNINGS`
+environment variable to ``"default"`` (or ``"d"``) before running
+Python.  Python code can also re-enable them
+by calling ``warnings.simplefilter('default')``.
+
+The ``unittest`` module also automatically reenables deprecation warnings
+when running tests.
 
 
 Python 3.1 Features
@@ -2464,6 +2510,54 @@ For applications that embed Python:
 .. ======================================================================
 
 
+.. _py27-maintenance-enhancements:
+
+New Features Added to Python 2.7 Maintenance Releases
+=====================================================
+
+New features may be added to Python 2.7 maintenance releases when the
+situation genuinely calls for it. Any such additions must go through
+the Python Enhancement Proposal process, and make a compelling case for why
+they can't be adequately addressed by either adding the new feature solely to
+Python 3, or else by publishing it on the Python Package Index.
+
+In addition to the specific proposals listed below, there is a general
+exemption allowing new ``-3`` warnings to be added in any Python 2.7
+maintenance release.
+
+
+PEP 434: IDLE Enhancement Exception for All Branches
+----------------------------------------------------
+
+:pep:`434` describes a general exemption for changes made to the IDLE
+development environment shipped along with Python. This exemption makes it
+possible for the IDLE developers to provide a more consistent user
+experience across all supported versions of Python 2 and 3.
+
+For details of any IDLE changes, refer to the NEWS file for the specific
+release.
+
+
+PEP 466: Network Security Enhancements for Python 2.7
+-----------------------------------------------------
+
+:pep:`466` describes a number of network security enhancement proposals
+that have been approved for inclusion in Python 2.7 maintenance releases,
+with the first of those changes appearing in the Python 2.7.7 release.
+
+:pep:`466` related features added in Python 2.7.7:
+
+* :func:`hmac.compare_digest` was added to make a timing attack resistant
+  comparison operation broadly available to Python 2 applications
+  (backported by Alex Gaynor in :issue:`21306`)
+
+* the version of OpenSSL linked with the prebuilt Windows installers
+  published on python.org was updated to 1.0.1g (contributed by
+  Zachary Ware in :issue:`21462`)
+
+
+.. ======================================================================
+
 .. _acks27:
 
 Acknowledgements

Lib/idlelib/idle_test/test_textview.py

-'''Test the functions and main class method of textView.py.
-
-Since all methods and functions create (or destroy) a TextViewer, which
-is a widget containing multiple widgets, all tests must be gui tests.
-Using mock Text would not change this.  Other mocks are used to retrieve
-information about calls.
-
-The coverage is essentially 100%.
-'''
-from test.test_support import requires
-requires('gui')
+'''Test the functions and main class method of textView.py.'''
 
 import unittest
 import os
+from test.test_support import requires
 from Tkinter import Tk, Text, TclError
 from idlelib import textView as tv
 from idlelib.idle_test.mock_idle import Func
 from idlelib.idle_test.mock_tk import Mbox
 
-def setUpModule():
-    global root
-    root = Tk()
-
-def tearDownModule():
-    global root
-    root.destroy()
-    del root
+orig_mbox = tv.tkMessageBox
 
+class textviewClassTest(unittest.TestCase):
 
-class TV(tv.TextViewer):  # used by TextViewTest
-    transient = Func()
-    grab_set = Func()
-    wait_window = Func()
+    @classmethod
+    def setUpClass(cls):
+        requires('gui')
+        cls.root = Tk()
+        cls.TV = TV = tv.TextViewer
+        TV.transient = Func()
+        TV.grab_set = Func()
+        TV.wait_window = Func()
 
-class TextViewTest(unittest.TestCase):
+    @classmethod
+    def tearDownClass(cls):
+        cls.root.destroy()
+        TV = cls.TV
+        del cls.root, cls.TV
+        del TV.transient, TV.grab_set, TV.wait_window
 
     def setUp(self):
+        TV = self.TV
         TV.transient.__init__()
         TV.grab_set.__init__()
         TV.wait_window.__init__()
 
+
     def test_init_modal(self):
-        view = TV(root, 'Title', 'test text')
+        TV = self.TV
+        view = TV(self.root, 'Title', 'test text')
         self.assertTrue(TV.transient.called)
         self.assertTrue(TV.grab_set.called)
         self.assertTrue(TV.wait_window.called)
         view.Ok()
 
     def test_init_nonmodal(self):
-        view = TV(root, 'Title', 'test text', modal=False)
+        TV = self.TV
+        view = TV(self.root, 'Title', 'test text', modal=False)
         self.assertFalse(TV.transient.called)
         self.assertFalse(TV.grab_set.called)
         self.assertFalse(TV.wait_window.called)
         view.Ok()
 
     def test_ok(self):
-        view = TV(root, 'Title', 'test text', modal=False)
+        view = self.TV(self.root, 'Title', 'test text', modal=False)
         view.destroy = Func()
         view.Ok()
         self.assertTrue(view.destroy.called)
@@ -66,32 +64,35 @@ class textviewTest(unittest.TestCase):
 
     @classmethod
     def setUpClass(cls):
-        cls.orig_mbox = tv.tkMessageBox
+        requires('gui')
+        cls.root = Tk()
         tv.tkMessageBox = Mbox
 
     @classmethod
     def tearDownClass(cls):
-        tv.tkMessageBox = cls.orig_mbox
-        del cls.orig_mbox
+        cls.root.destroy()
+        del cls.root
+        tv.tkMessageBox = orig_mbox
 
     def test_view_text(self):
         # If modal True, tkinter will error with 'can't invoke "event" command'
-        view = tv.view_text(root, 'Title', 'test text', modal=False)
+        view = tv.view_text(self.root, 'Title', 'test text', modal=False)
         self.assertIsInstance(view, tv.TextViewer)
 
     def test_view_file(self):
         test_dir = os.path.dirname(__file__)
         testfile = os.path.join(test_dir, 'test_textview.py')
-        view = tv.view_file(root, 'Title', testfile, modal=False)
+        view = tv.view_file(self.root, 'Title', testfile, modal=False)
         self.assertIsInstance(view, tv.TextViewer)
         self.assertIn('Test', view.textView.get('1.0', '1.end'))
         view.Ok()
 
         # Mock messagebox will be used and view_file will not return anything
         testfile = os.path.join(test_dir, '../notthere.py')
-        view = tv.view_file(root, 'Title', testfile, modal=False)
+        view = tv.view_file(self.root, 'Title', testfile, modal=False)
         self.assertIsNone(view)
 
-
 if __name__ == '__main__':
-    unittest.main(verbosity=2)
+    unittest.main(verbosity=2, exit=False)
+    from idlelib.idle_test.htest import run
+    run(TextViewer)

Lib/pydoc.py

@@ -581,10 +581,15 @@ class HTMLDoc(Doc):
             elif pep:
                 url = 'http://www.python.org/dev/peps/pep-%04d/' % int(pep)
                 results.append('<a href="%s">%s</a>' % (url, escape(all)))
+            elif selfdot:
+                # Create a link for methods like 'self.method(...)'
+                # and use <strong> for attributes like 'self.attr'
+                if text[end:end+1] == '(':
+                    results.append('self.' + self.namelink(name, methods))
+                else:
+                    results.append('self.<strong>%s</strong>' % name)
             elif text[end:end+1] == '(':
                 results.append(self.namelink(name, methods, funcs, classes))
-            elif selfdot:
-                results.append('self.<strong>%s</strong>' % name)
             else:
                 results.append(self.namelink(name, classes))
             here = end
@@ -1372,6 +1377,8 @@ def getpager():
     """Decide what method to use for paging through text."""
     if type(sys.stdout) is not types.FileType:
         return plainpager
+    if not hasattr(sys.stdin, "isatty"):
+        return plainpager
     if not sys.stdin.isatty() or not sys.stdout.isatty():
         return plainpager
     if 'PAGER' in os.environ:

Lib/test/pydoc_mod.py

@@ -15,6 +15,16 @@ class B(object):
     NO_MEANING = "eggs"
     pass
 
+class C(object):
+    def say_no(self):
+        return "no"
+    def get_answer(self):
+        """ Return say_no() """
+        return self.say_no()
+    def is_it_true(self):
+        """ Return self.get_answer() """
+        return self.get_answer()
+
 def doc_func():
     """
     This function solves all of the world's problems:

Lib/test/test_grammar.py

@@ -75,6 +75,12 @@ class TokenTests(unittest.TestCase):
         x = .3e14
         x = 3.1e4
 
+    def test_float_exponent_tokenization(self):
+        # See issue 21642.
+        self.assertEqual(1 if 1else 0, 1)
+        self.assertEqual(1 if 0else 0, 0)
+        self.assertRaises(SyntaxError, eval, "0 if 1Else 0")
+
     def testStringLiterals(self):
         x = ''; y = ""; self.assertTrue(len(x) == 0 and x == y)
         x = '\''; y = "'"; self.assertTrue(len(x) == 1 and x == y and ord(x) == 39)

Lib/test/test_pydoc.py

@@ -38,6 +38,7 @@ FILE
 CLASSES
     __builtin__.object
         B
+        C
     A
 \x20\x20\x20\x20
     class A
@@ -59,6 +60,26 @@ CLASSES
      |  Data and other attributes defined here:
      |\x20\x20
      |  NO_MEANING = 'eggs'
+\x20\x20\x20\x20
+    class C(__builtin__.object)
+     |  Methods defined here:
+     |\x20\x20
+     |  get_answer(self)
+     |      Return say_no()
+     |\x20\x20
+     |  is_it_true(self)
+     |      Return self.get_answer()
+     |\x20\x20
+     |  say_no(self)
+     |\x20\x20
+     |  ----------------------------------------------------------------------
+     |  Data descriptors defined here:
+     |\x20\x20
+     |  __dict__
+     |      dictionary for instance variables (if defined)
+     |\x20\x20
+     |  __weakref__
+     |      list of weak references to the object (if defined)
 
 FUNCTIONS
     doc_func()
@@ -108,6 +129,7 @@ expected_html_pattern = \
 </font></dt><dd>
 <dl>
 <dt><font face="helvetica, arial"><a href="test.pydoc_mod.html#B">B</a>
+</font></dt><dt><font face="helvetica, arial"><a href="test.pydoc_mod.html#C">C</a>
 </font></dt></dl>
 </dd>
 <dt><font face="helvetica, arial"><a href="test.pydoc_mod.html#A">A</a>
@@ -142,6 +164,28 @@ expected_html_pattern = \
 Data and other attributes defined here:<br>
 <dl><dt><strong>NO_MEANING</strong> = 'eggs'</dl>
 
+</td></tr></table> <p>
+<table width="100%%" cellspacing=0 cellpadding=2 border=0 summary="section">
+<tr bgcolor="#ffc8d8">
+<td colspan=3 valign=bottom>&nbsp;<br>
+<font color="#000000" face="helvetica, arial"><a name="C">class <strong>C</strong></a>(<a href="__builtin__.html#object">__builtin__.object</a>)</font></td></tr>
+\x20\x20\x20\x20
+<tr><td bgcolor="#ffc8d8"><tt>&nbsp;&nbsp;&nbsp;</tt></td><td>&nbsp;</td>
+<td width="100%%">Methods defined here:<br>
+<dl><dt><a name="C-get_answer"><strong>get_answer</strong></a>(self)</dt><dd><tt>Return&nbsp;<a href="#C-say_no">say_no</a>()</tt></dd></dl>
+
+<dl><dt><a name="C-is_it_true"><strong>is_it_true</strong></a>(self)</dt><dd><tt>Return&nbsp;self.<a href="#C-get_answer">get_answer</a>()</tt></dd></dl>
+
+<dl><dt><a name="C-say_no"><strong>say_no</strong></a>(self)</dt></dl>
+
+<hr>
+Data descriptors defined here:<br>
+<dl><dt><strong>__dict__</strong></dt>
+<dd><tt>dictionary&nbsp;for&nbsp;instance&nbsp;variables&nbsp;(if&nbsp;defined)</tt></dd>
+</dl>
+<dl><dt><strong>__weakref__</strong></dt>
+<dd><tt>list&nbsp;of&nbsp;weak&nbsp;references&nbsp;to&nbsp;the&nbsp;object&nbsp;(if&nbsp;defined)</tt></dd>
+</dl>
 </td></tr></table></td></tr></table><p>
 <table width="100%%" cellspacing=0 cellpadding=2 border=0 summary="section">
 <tr bgcolor="#eeaa77">
@@ -289,6 +333,14 @@ class PydocDocTest(unittest.TestCase):
         result, doc_loc = get_pydoc_text(xml.etree)
         self.assertEqual(doc_loc, "", "MODULE DOCS incorrectly includes a link")
 
+    def test_getpager_with_stdin_none(self):
+        previous_stdin = sys.stdin
+        try:
+            sys.stdin = None
+            pydoc.getpager() # Shouldn't fail.
+        finally:
+            sys.stdin = previous_stdin
+
     def test_non_str_name(self):
         # issue14638
         # Treat illegal (non-str) name like no name

Misc/ACKS

@@ -194,6 +194,7 @@ Roger Burnham
 Alastair Burt
 Tarn Weisner Burton
 Lee Busby
+Katherine Busch
 Ralph Butler
 Nicolas Cadou
 Jp Calderone
@@ -749,6 +750,7 @@ Julia Lawall
 Chris Lawrence
 Brian Leair
 Mathieu Leduc-Hamel
+Amandine Lee
 Christopher Lee
 Inyeol Lee
 James Lee

Misc/NEWS

@@ -13,6 +13,10 @@ Core and Builtins
 - Issue #19656: Running Python with the -3 option now also warns about
   non-ascii bytes literals.
 
+- Issue #21642: If the conditional if-else expression, allow an integer written
+  with no space between itself and the ``else`` keyword (e.g. ``True if 42else
+  False``) to be valid syntax.
+
 - Issue #21523: Fix over-pessimistic computation of the stack effect of
   some opcodes in the compiler.  This also fixes a quadratic compilation
   time issue noticeable when compiling code with a large number of "and"
@@ -24,6 +28,12 @@ Library
 - Issue #21304: Backport the key derivation function hashlib.pbkdf2_hmac from
   Python 3 per PEP 466.
 
+- Issue #11709: Fix the pydoc.help function to not fail when sys.stdin is not a
+  valid file.
+
+- Issue #13223: Fix pydoc.writedoc so that the HTML documentation for methods
+  that use 'self' in the example code is generated correctly.
+
 - Issue #21552: Fixed possible integer overflow of too long string lengths in
   the tkinter module on 64-bit platforms.
 

Parser/tokenizer.c

@@ -1500,15 +1500,24 @@ tok_get(register struct tok_state *tok, char **p_start, char **p_end)
                     } while (isdigit(c));
                 }
                 if (c == 'e' || c == 'E') {
-        exponent:
+                    int e;
+                  exponent:
+                    e = c;
                     /* Exponent part */
                     c = tok_nextc(tok);
-                    if (c == '+' || c == '-')
+                    if (c == '+' || c == '-') {
                         c = tok_nextc(tok);
-                    if (!isdigit(c)) {
-                        tok->done = E_TOKEN;
+                        if (!isdigit(c)) {
+                            tok->done = E_TOKEN;
+                            tok_backup(tok, c);
+                            return ERRORTOKEN;
+                        }
+                    } else if (!isdigit(c)) {
                         tok_backup(tok, c);
-                        return ERRORTOKEN;
+                        tok_backup(tok, e);
+                        *p_start = tok->start;
+                        *p_end = tok->cur;
+                        return NUMBER;
                     }
                     do {
                         c = tok_nextc(tok);