Draft: Implement PEP-0451 ModuleSpec and use python 3.3 per-module lock rather than Global Import Lock (!2209) · Merge requests · nexedi / erp5

Draft: Implement PEP-0451 ModuleSpec and use python 3.3 per-module lock rather than Global Import Lock

Since Python 3.3, there is a nice per-module lock implementation (with DeadLock avoidance mechanism) instead of the Global Import Lock so we don't have to fiddle anymore with Global Import Lock to prevent a DeadLock with ZEO.

In Python 3.4, PEP-0451 has been implemented and provides a new ModuleSpec object for Import Hooks along with a new API to load modules (through Loader implementing create_module() and exec_module()). This provides the boilerplate to add module to its parent and sys.modules among other things. This deprecates PEP-0302 implementation we currently have (find_module() and load_module()) since Python 3.10, find_module() having been removed in Python 3.12.

This MR rewrites most of ZODB Component implementation to implement per-module lock and PEP-0451. Of course, I could have just implemented find_spec() by calling find_module() but this would have not taken advantage of per-module lock and the new API, so I thought it better to rewrite most of the code, at the same time doing a cleanup since the first design of ZODB Components where it was not completely clear at that time where this would lead up.

I documented everything in the comments of component_package.py.

Here is a TODO-list before it can actually be merged (besides of a nice commit message of course):

aq_method_lock is not acquired: not sure yet where this should be done. I was thinking of monkey-patching Lib/importlib/_bootstrap.py:_ModuleLockManager to acquire the lock on __enter__() instead of doing an acquire/release on each create_module() and exec_module() calls. I haven't thought so much about it yet...
Call of createFilesystemImportDict() is not done and thus filesystem import compatibility does not work. This is a minor issue I'm going to fix soon.
Implement an Unit Test for ERP5_COMPONENT_OVERRIDE_PATH (can be done after the merge as this does not exist yet anyway).
Fix Unit Tests failures (I haven't ran all Unit Tests yet but I have 2-3 tests to fix in testDynamicClassGeneration).
Do we really need find_load_module() as I now raise a ComponentImportError when a Component could not found or when exec() fails? Maybe yet another ComponentModuleNotFoundError could be raised to for find_load_module() use case?