GitLab

Python3 chains exceptions, so that e.g. if exc1 is raised and, while it
was not handled, another exc2 is raised, exc2 will be linked to exc1 via
exc2.__context__ attribute and exc1 will be included into exc2 traceback
printout. However many projects still use Python2 and there is no
similar chaining functionality there. This way exc1 is completely lost.

Since defer code is in our hands, we can teach it to implement exception
chaining even on Python2 by carefully analyzing what happens in
_GoFrame.__exit__().

Implementing chaining itself is relatively easy, but is only part of the
story. Even if an exception is chained with its cause, but exception
dump does not show the cause, the chaining will be practically useless.
With this in mind this patches settles not only on implementing chaining
itself, but on also giving a promise that chained cause exceptions will
be included into traceback dumps as well.

To realize this promise we adjust all exception dumping funcitons in
traceback module and carefully install adjusted
traceback.print_exception() into sys.excepthook. This amends python
interactive sessions and programs run by python interpreter to include
causes in exception dumps. "Careful" here means that we don't change
sys.excepthook if on golang module load we see that sys.excepthook was already
changed by some other module - e.g. due to IPython session running
because IPython installs its own sys.excepthook. In such cases we don't
install our sys.excepthook, but we also provide integration patches that
add exception chaining support for traceback dump functionality in
popular third-party software. The patches (currently for IPython and
Pytest) are activated automatically, but only when/if corresponding
software is imported and actually used. This should give practically
good implementation of the promise - a user can now rely on seeing
exception cause in traceback dump whatever way python programs are run.

The implementation takes https://pypi.org/project/pep3134/ experience
into account [1]. peak.utils.imports [2,3] is used to be notified when/if
third-party module is imported.

[1] https://github.com/9seconds/pep3134/
[2] https://pypi.org/project/Importing/
[3] http://peak.telecommunity.com/DevCenter/Importing

This patch originally started as hacky workaround in wendelin.core
because in wcfs tests I was frequently hitting situations, where
exception raised by an assert was hidden by another exception raised in
further generic teardown check. For example wcfs tests check that wcfs
is unmounted after every test run [4] and if that fails it was hiding
problems raised by an assert. As the result I was constantly guessing
and adding code like [5] to find what was actually breaking. At some
point I added hacky workaround for defer to print cause exception not to
loose it [6]. [7] has more context and background discussion on this topic.

[4] https://lab.nexedi.com/kirr/wendelin.core/blob/49e73a6d/wcfs/wcfs_test.py#L70
[5] https://lab.nexedi.com/kirr/wendelin.core/blob/49e73a6d/wcfs/wcfs_test.py#L853-857
[6] kirr/wendelin.core@c00d94c7
[7] zodbtools!13 (comment 81553)

After this patch, on Python2

    defer(cleanup1)
    defer(cleanup2)
    defer(cleanup3)
    ...

is no longer just a syntatic sugar for

    try:
        try:
            try:
                ...
            finally:
                cleanup3()
        finally:
            cleanup2()
    finally:
        cleanup1()