Commit 9f4dde50 authored by Kirill Smelkov's avatar Kirill Smelkov

pylint: Fix crash on LazyModules if those modules are not actually available

Pygolang installs import hooks for pytest and ipython, to add
exception-chaining support into them(*) _iff_ (if and only if) those
modules are actually used. This works via Importing[1] which
pre-installs artificial modules into sys.modules that catch
__getattribute__ and try to import corresponding module for real on
first access. Usually everything is fine.

But with pylint/astroid, if the checker code happens to run with those
LazyModules installed, and the checked code has `import sys` somewhere,
astroid eventually delves into processing sys, then sys.modules and
wants to represent that sys.modules dict as dict of constant. Then, when
e.g. sys.modules['_pytest'] is processed, corresponding module object is
checked for .__class__, which raises ImportError if pytest is not
actually available:

    ( https://erp5js.nexedi.net/#/test_result_module/20220127-129289AE2/33 )
    ...
    File ".../eggs/astroid-1.3.8-py2.7.egg/astroid/node_classes.py", line 553, in __init__
        for k, v in items.items()]
      File ".../eggs/astroid-1.3.8-py2.7.egg/astroid/node_classes.py", line 962, in const_factory
        return CONST_CLS[value.__class__](value)
      File ".../eggs/Importing-1.10-py2.7.egg/peak/util/imports.py", line 254, in __getattribute__
        _loadModule(self)
      File ".../eggs/Importing-1.10-py2.7.egg/peak/util/imports.py", line 222, in _loadModule
        reload(module)
    ImportError: No module named _pytest

-> Fix it by detecting those lazy modules and not letting them go through
normal const_factory not to crash.

/cc @jerome, @arnau
/reviewed-by @rafael
/reviewed-on nexedi/erp5!1546 and nexedi/slapos!1117

[1] https://pypi.org/project/Importing/

(*) see:
    https://lab.nexedi.com/nexedi/pygolang/blob/pygolang-0.1-0-g7b72d41/golang/_patch/__init__.py
    https://lab.nexedi.com/nexedi/pygolang/blob/pygolang-0.1-0-g7b72d41/golang/_patch/pytest_py2.py#L48-51
    https://lab.nexedi.com/nexedi/pygolang/blob/pygolang-0.1-0-g7b72d41/golang/_patch/ipython_py2.py#L45-48
parent f4c20e20
Pipeline #19509 failed with stage
in 0 seconds
......@@ -39,7 +39,7 @@ OptionsManagerMixIn.read_config_file = lambda *args, **kw: None
## Pylint transforms and plugin to generate AST for ZODB Components
from astroid.builder import AstroidBuilder
from astroid.exceptions import AstroidBuildingException
from astroid import MANAGER
from astroid import MANAGER, node_classes
try:
from astroid.builder import _guess_encoding
......@@ -98,6 +98,32 @@ def string_build(self, data, modname='', path=None):
return self._post_build(module, encoding)
AstroidBuilder.string_build = string_build
# patch node_classes.const_factory not to fail on LazyModules that e.g.
# pygolang installs for pytest and ipython into sys.modules dict:
#
# https://lab.nexedi.com/nexedi/pygolang/blob/pygolang-0.1-0-g7b72d41/golang/_patch/__init__.py
# https://lab.nexedi.com/nexedi/pygolang/blob/pygolang-0.1-0-g7b72d41/golang/_patch/pytest_py2.py#L48-51
# https://lab.nexedi.com/nexedi/pygolang/blob/pygolang-0.1-0-g7b72d41/golang/_patch/ipython_py2.py#L45-48
#
# if we don't patch and the module is not available, upon checking sys->sys.modules
# const_factory will fail with ImportError when accessing value.__class__.
node_classes_const_factory = node_classes.const_factory
def const_factory(value):
typ = type(value)
typename = ('%s.%s' % (typ.__module__, typ.__name__))
if typename == 'peak.util.imports.LazyModule':
# lazy module installed by Importing
# see if we can load it, and return empty placehoder if the module is not available
try:
value.__class__
except ImportError:
node = node_classes.EmptyNode()
node.object = None # not value
return node
# ok the module is available and is now loaded - continue via normal const_factory path
return node_classes_const_factory(value)
node_classes.const_factory = const_factory
from astroid import nodes
def erp5_package_transform(node):
"""'
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment