Close issue #6210: Implement PEP 409

Doc/ACKS.txt

@@ -62,6 +62,7 @@ docs@python.org), and we'll be glad to correct the problem.
    * Stefan Franke
    * Jim Fulton
    * Peter Funk
+   * Ethan Furman
    * Lele Gaifax
    * Matthew Gallagher
    * Gabriel Genellina

Doc/c-api/exceptions.rst

@@ -421,17 +421,24 @@ Exception Objects
 
 .. c:function:: PyObject* PyException_GetCause(PyObject *ex)
 
-   Return the cause (another exception instance set by ``raise ... from ...``)
-   associated with the exception as a new reference, as accessible from Python
-   through :attr:`__cause__`.  If there is no cause associated, this returns
-   *NULL*.
+   Return the cause (either an exception instance, or :const:`None`,
+   set by ``raise ... from ...``) associated with the exception as a new
+   reference, as accessible from Python through :attr:`__cause__`.
+
+   If there is no cause associated, this returns *NULL* (from Python
+   ``__cause__ is Ellipsis``).  If the cause is :const:`None`, the default
+   exception display routines stop showing the context chain.
 
 
 .. c:function:: void PyException_SetCause(PyObject *ex, PyObject *ctx)
 
    Set the cause associated with the exception to *ctx*.  Use *NULL* to clear
-   it.  There is no type check to make sure that *ctx* is an exception instance.
-   This steals a reference to *ctx*.
+   it.  There is no type check to make sure that *ctx* is either an exception
+   instance or :const:`None`.  This steals a reference to *ctx*.
+
+   If the cause is set to :const:`None` the default exception display
+   routines will not display this exception's context, and will not follow the
+   chain any further.
 
 
 .. _unicodeexceptions:

Doc/library/exceptions.rst

@@ -34,6 +34,24 @@ programmers are encouraged to at least derive new exceptions from the
 defining exceptions is available in the Python Tutorial under
 :ref:`tut-userexceptions`.
 
+When raising (or re-raising) an exception in an :keyword:`except` clause
+:attr:`__context__` is automatically set to the last exception caught; if the
+new exception is not handled the traceback that is eventually displayed will
+include the originating exception(s) and the final exception.
+
+This implicit exception chain can be made explicit by using :keyword:`from`
+with :keyword:`raise`.  The single argument to :keyword:`from` must be an
+exception or :const:`None`, and it will bet set as :attr:`__cause__` on the
+raised exception.  If :attr:`__cause__` is an exception it will be displayed
+instead of :attr:`__context__`; if :attr:`__cause__` is None,
+:attr:`__context__` will not be displayed by the default exception handling
+code.  (Note:  the default value for :attr:`__context__` is :const:`None`,
+while the default value for :attr:`__cause__` is :const:`Ellipsis`.)
+
+In either case, the default exception handling code will not display
+any of the remaining links in the :attr:`__context__` chain if
+:attr:`__cause__` has been set.
+
 
 Base classes
 ------------

Doc/library/stdtypes.rst

@@ -2985,10 +2985,11 @@ It is written as ``None``.
 The Ellipsis Object
 -------------------
 
-This object is commonly used by slicing (see :ref:`slicings`).  It supports no
-special operations.  There is exactly one ellipsis object, named
-:const:`Ellipsis` (a built-in name).  ``type(Ellipsis)()`` produces the
-:const:`Ellipsis` singleton.
+This object is commonly used by slicing (see :ref:`slicings`), but may also
+be used in other situations where a sentinel value other than :const:`None`
+is needed.  It supports no special operations.  There is exactly one ellipsis
+object, named :const:`Ellipsis` (a built-in name).  ``type(Ellipsis)()``
+produces the :const:`Ellipsis` singleton.
 
 It is written as ``Ellipsis`` or ``...``.
 

Doc/whatsnew/3.3.rst

@@ -254,6 +254,9 @@ inspection of exception attributes::
 PEP 380: Syntax for Delegating to a Subgenerator
 ================================================
 
+:pep:`380` - Syntax for Delegating to a Subgenerator
+ PEP written by Greg Ewing.
+
 PEP 380 adds the ``yield from`` expression, allowing a generator to delegate
 part of its operations to another generator. This allows a section of code
 containing 'yield' to be factored out and placed in another generator.
@@ -267,6 +270,67 @@ Kelly and Nick Coghlan, documentation by Zbigniew Jędrzejewski-Szmek and
 Nick Coghlan)
 
 
+PEP 409: Suppressing exception context
+======================================
+
+:pep:`409` - Suppressing exception context
+ PEP written by Ethan Furman, implemented by Ethan Furman and Nick Coghlan.
+
+PEP 409 introduces new syntax that allows the display of the chained
+exception context to be disabled. This allows cleaner error messages in
+applications that convert between exception types::
+
+    >>> class D:
+    ...     def __init__(self, extra):
+    ...         self._extra_attributes = extra
+    ...     def __getattr__(self, attr):
+    ...         try:
+    ...             return self._extra_attributes[attr]
+    ...         except KeyError:
+    ...             raise AttributeError(attr) from None
+    ...
+    >>> D({}).x
+    Traceback (most recent call last):
+      File "<stdin>", line 1, in <module>
+      File "<stdin>", line 8, in __getattr__
+    AttributeError: x
+
+Without the ``from None`` suffix to suppress the cause, the original
+exception would be displayed by default::
+
+    >>> class C:
+    ...     def __init__(self, extra):
+    ...         self._extra_attributes = extra
+    ...     def __getattr__(self, attr):
+    ...         try:
+    ...             return self._extra_attributes[attr]
+    ...         except KeyError:
+    ...             raise AttributeError(attr)
+    ...
+    >>> C({}).x
+    Traceback (most recent call last):
+      File "<stdin>", line 6, in __getattr__
+    KeyError: 'x'
+
+    During handling of the above exception, another exception occurred:
+
+    Traceback (most recent call last):
+      File "<stdin>", line 1, in <module>
+      File "<stdin>", line 8, in __getattr__
+    AttributeError: x
+
+No debugging capability is lost, as the original exception context remains
+available if needed (for example, if an intervening library has incorrectly
+suppressed valuable underlying details)::
+
+    >>> try:
+    ...     D({}).x
+    ... except AttributeError as exc:
+    ...     print(repr(exc.__context__))
+    ...
+    KeyError('x',)
+
+
 PEP 3155: Qualified name for classes and functions
 ==================================================
 

Include/pyerrors.h

@@ -105,6 +105,7 @@ PyAPI_FUNC(PyObject *) PyException_GetTraceback(PyObject *);
 /* Cause manipulation (PEP 3134) */
 PyAPI_FUNC(PyObject *) PyException_GetCause(PyObject *);
 PyAPI_FUNC(void) PyException_SetCause(PyObject *, PyObject *);
+PyAPI_FUNC(int) _PyException_SetCauseChecked(PyObject *, PyObject *);
 
 /* Context manipulation (PEP 3134) */
 PyAPI_FUNC(PyObject *) PyException_GetContext(PyObject *);

Lib/test/test_exceptions.py

@@ -387,19 +387,36 @@ class ExceptionTests(unittest.TestCase):
 
     def testChainingAttrs(self):
         e = Exception()
-        self.assertEqual(e.__context__, None)
-        self.assertEqual(e.__cause__, None)
+        self.assertIsNone(e.__context__)
+        self.assertIs(e.__cause__, Ellipsis)
 
         e = TypeError()
-        self.assertEqual(e.__context__, None)
-        self.assertEqual(e.__cause__, None)
+        self.assertIsNone(e.__context__)
+        self.assertIs(e.__cause__, Ellipsis)
 
         class MyException(EnvironmentError):
             pass
 
         e = MyException()
-        self.assertEqual(e.__context__, None)
-        self.assertEqual(e.__cause__, None)
+        self.assertIsNone(e.__context__)
+        self.assertIs(e.__cause__, Ellipsis)
+
+    def testChainingDescriptors(self):
+        try:
+            raise Exception()
+        except Exception as exc:
+            e = exc
+
+        self.assertIsNone(e.__context__)
+        self.assertIs(e.__cause__, Ellipsis)
+
+        e.__context__ = NameError()
+        e.__cause__ = None
+        self.assertIsInstance(e.__context__, NameError)
+        self.assertIsNone(e.__cause__)
+
+        e.__cause__ = Ellipsis
+        self.assertIs(e.__cause__, Ellipsis)
 
     def testKeywordArgs(self):
         # test that builtin exception don't take keyword args,

Lib/test/test_raise.py

@@ -3,12 +3,27 @@
 
 """Tests for the raise statement."""
 
-from test import support
+from test import support, script_helper
+import re
 import sys
 import types
 import unittest
 
 
+try:
+    from resource import setrlimit, RLIMIT_CORE, error as resource_error
+except ImportError:
+    prepare_subprocess = None
+else:
+    def prepare_subprocess():
+        # don't create core file
+        try:
+            setrlimit(RLIMIT_CORE, (0, 0))
+        except (ValueError, resource_error):
+            pass
+
+
+
 def get_tb():
     try:
         raise OSError()
@@ -77,6 +92,16 @@ class TestRaise(unittest.TestCase):
                 nested_reraise()
         self.assertRaises(TypeError, reraise)
 
+    def test_raise_from_None(self):
+        try:
+            try:
+                raise TypeError("foo")
+            except:
+                raise ValueError() from None
+        except ValueError as e:
+            self.assertTrue(isinstance(e.__context__, TypeError))
+            self.assertIsNone(e.__cause__)
+
     def test_with_reraise1(self):
         def reraise():
             try:
@@ -139,6 +164,23 @@ class TestRaise(unittest.TestCase):
 
 
 class TestCause(unittest.TestCase):
+
+    def testCauseSyntax(self):
+        try:
+            try:
+                try:
+                    raise TypeError
+                except Exception:
+                    raise ValueError from None
+            except ValueError as exc:
+                self.assertIsNone(exc.__cause__)
+                raise exc from Ellipsis
+        except ValueError as exc:
+            e = exc
+
+        self.assertIs(e.__cause__, Ellipsis)
+        self.assertIsInstance(e.__context__, TypeError)
+
     def test_invalid_cause(self):
         try:
             raise IndexError from 5
@@ -178,6 +220,44 @@ class TestCause(unittest.TestCase):
 
 
 class TestTraceback(unittest.TestCase):
+
+    def get_output(self, code, filename=None):
+        """
+        Run the specified code in Python (in a new child process) and read the
+        output from the standard error or from a file (if filename is set).
+        Return the output lines as a list.
+        """
+        options = {}
+        if prepare_subprocess:
+            options['preexec_fn'] = prepare_subprocess
+        process = script_helper.spawn_python('-c', code, **options)
+        stdout, stderr = process.communicate()
+        exitcode = process.wait()
+        output = support.strip_python_stderr(stdout)
+        output = output.decode('ascii', 'backslashreplace')
+        if filename:
+            self.assertEqual(output, '')
+            with open(filename, "rb") as fp:
+                output = fp.read()
+            output = output.decode('ascii', 'backslashreplace')
+        output = re.sub('Current thread 0x[0-9a-f]+',
+                        'Current thread XXX',
+                        output)
+        return output.splitlines(), exitcode
+
+    def test_traceback_verbiage(self):
+        code = """
+try:
+    raise ValueError
+except:
+    raise NameError from None
+"""
+        text, exitcode = self.get_output(code)
+        self.assertEqual(len(text), 3)
+        self.assertTrue(text[0].startswith('Traceback'))
+        self.assertTrue(text[1].startswith('  File '))
+        self.assertTrue(text[2].startswith('NameError'))
+
     def test_sets_traceback(self):
         try:
             raise IndexError()

Lib/test/test_traceback.py

@@ -246,6 +246,21 @@ class BaseExceptionReportingTests:
         self.check_zero_div(blocks[0])
         self.assertIn('inner_raise() # Marker', blocks[2])
 
+    def test_context_suppression(self):
+        try:
+            try:
+                raise Exception
+            except:
+                raise ZeroDivisionError from None
+        except ZeroDivisionError as _:
+            e = _
+        lines = self.get_report(e).splitlines()
+        self.assertEqual(len(lines), 4)
+        self.assertTrue(lines[0].startswith('Traceback'))
+        self.assertTrue(lines[1].startswith('  File'))
+        self.assertIn('ZeroDivisionError from None', lines[2])
+        self.assertTrue(lines[3].startswith('ZeroDivisionError'))
+
     def test_cause_and_context(self):
         # When both a cause and a context are set, only the cause should be
         # displayed and the context should be muted.

Lib/traceback.py

@@ -120,14 +120,14 @@ def _iter_chain(exc, custom_tb=None, seen=None):
     seen.add(exc)
     its = []
     cause = exc.__cause__
-    if cause is not None and cause not in seen:
-        its.append(_iter_chain(cause, None, seen))
-        its.append([(_cause_message, None)])
-    else:
+    if cause is Ellipsis:
         context = exc.__context__
         if context is not None and context not in seen:
             its.append(_iter_chain(context, None, seen))
             its.append([(_context_message, None)])
+    elif cause is not None and cause not in seen:
+        its.append(_iter_chain(cause, False, seen))
+        its.append([(_cause_message, None)])
     its.append([(exc, custom_tb or exc.__traceback__)])
     # itertools.chain is in an extension module and may be unavailable
     for it in its:

Misc/ACKS

@@ -338,6 +338,7 @@ Jim Fulton
 Tadayoshi Funaba
 Gyro Funch
 Peter Funk
+Ethan Furman
 Geoff Furnish
 Ulisses Furquim
 Hagen Fürstenau

Misc/NEWS

@@ -10,6 +10,10 @@ What's New in Python 3.3 Alpha 1?
 Core and Builtins
 -----------------
 
+- PEP 409, Issue #6210: "raise X from None" is now supported as a means of
+  suppressing the display of the chained exception context. The chained
+  context still remains available as the __context__ attribute.
+
 - Issue #10181: New memoryview implementation fixes multiple ownership
   and lifetime issues of dynamically allocated Py_buffer members (#9990)
   as well as crashes (#8305, #7433). Many new features have been added

Objects/exceptions.c

@@ -266,28 +266,35 @@ BaseException_get_cause(PyObject *self) {
     PyObject *res = PyException_GetCause(self);
     if (res)
         return res;  /* new reference already returned above */
-    Py_RETURN_NONE;
+    Py_INCREF(Py_Ellipsis);
+    return Py_Ellipsis;
 }
 
-static int
-BaseException_set_cause(PyObject *self, PyObject *arg) {
-    if (arg == NULL) {
-        PyErr_SetString(PyExc_TypeError, "__cause__ may not be deleted");
-        return -1;
-    } else if (arg == Py_None) {
+int
+_PyException_SetCauseChecked(PyObject *self, PyObject *arg) {
+    if (arg == Py_Ellipsis) {
         arg = NULL;
-    } else if (!PyExceptionInstance_Check(arg)) {
-        PyErr_SetString(PyExc_TypeError, "exception cause must be None "
-                        "or derive from BaseException");
+    } else if (arg != Py_None && !PyExceptionInstance_Check(arg)) {
+        PyErr_SetString(PyExc_TypeError, "exception cause must be None, "
+                        "Ellipsis or derive from BaseException");
         return -1;
     } else {
-        /* PyException_SetCause steals this reference */
+        /* PyException_SetCause steals a reference */
         Py_INCREF(arg);
     }
     PyException_SetCause(self, arg);
     return 0;
 }
 
+static int
+BaseException_set_cause(PyObject *self, PyObject *arg) {
+    if (arg == NULL) {
+        PyErr_SetString(PyExc_TypeError, "__cause__ may not be deleted");
+        return -1;
+    }
+    return _PyException_SetCauseChecked(self, arg);
+}
+
 
 static PyGetSetDef BaseException_getset[] = {
     {"__dict__", PyObject_GenericGetDict, PyObject_GenericSetDict},

Python/ceval.c

@@ -3567,22 +3567,23 @@ do_raise(PyObject *exc, PyObject *cause)
 
     if (cause) {
         PyObject *fixed_cause;
+        int result;
         if (PyExceptionClass_Check(cause)) {
             fixed_cause = PyObject_CallObject(cause, NULL);
             if (fixed_cause == NULL)
                 goto raise_error;
-            Py_DECREF(cause);
-        }
-        else if (PyExceptionInstance_Check(cause)) {
+            Py_CLEAR(cause);
+        } else {
+            /* Let "exc.__cause__ = cause" handle all further checks */
             fixed_cause = cause;
+            cause = NULL; /* Steal the reference */
         }
-        else {
-            PyErr_SetString(PyExc_TypeError,
-                            "exception causes must derive from "
-                            "BaseException");
+        /* We retain ownership of the reference to fixed_cause */
+        result = _PyException_SetCauseChecked(value, fixed_cause);
+        Py_DECREF(fixed_cause);
+        if (result < 0) {
             goto raise_error;
         }
-        PyException_SetCause(value, fixed_cause);
     }
 
     PyErr_SetObject(type, value);

Python/pythonrun.c

@@ -1698,7 +1698,11 @@ print_exception_recursive(PyObject *f, PyObject *value, PyObject *seen)
         else if (PyExceptionInstance_Check(value)) {
             cause = PyException_GetCause(value);
             context = PyException_GetContext(value);
-            if (cause) {
+            if (cause && cause == Py_None) {
+                /* print neither cause nor context */
+                ;
+            }
+            else if (cause) {
                 res = PySet_Contains(seen, cause);
                 if (res == -1)
                     PyErr_Clear();