• Jérome Perrin's avatar
    zodbrestore: Mark restore-with-extension tests as xfail on ZODB4 · aa7e1966
    Jérome Perrin authored
    @kirr wrote (!19 (comment 129442))
    
        For the reference - contrary to ZODB5, restore tests on ZODB4 are currently
        [broken](https://nexedijs.erp5.net/#/test_result_module/20210317-B3AC205A/2).
        Restored file is not bit-to-bit identical to the original.
    
        The problem is that on commit/restore, we need to save
        user/description/extension. For extension `zodbdump.Transaction` provides
        .extension_bytes, which ZODB5 uses to save its raw copy. However ZODB4 goes
        through `.extension` and pickles it:
    
        https://lab.nexedi.com/nexedi/zodbtools/blob/129afa67/zodbtools/zodbdump.py#L425-453
        https://github.com/zopefoundation/ZODB/blob/4/src/ZODB/BaseStorage.py#L220-L240
    
        This leads to unpickle-repickle round-trip and different extension being committed on restore:
    
        ```diff
        diff --git a/1zdump b/2zdump
        index 5033bc1..a3a32aa 100644
        --- a/1zdump
        +++ b/2zdump
        @@ -10,7 +10,7 @@ q^A.
         txn 0285cbac3d0369e6 " "
         user "user0.0"
         description "step 0.0"
        -extension "\x80\x02}q\x01(U\tx-cookieSU\x05RF9IEU\vx-generatorq\x02U\fzodb/py2 (f)u."
        +extension "}q\x01(U\tx-cookieSU\x05RF9IEU\vx-generatorU\fzodb/py2 (f)u."
         obj 0000000000000000 98 sha1:eba252d1984f975ecb636bc1b3a89c953dd20527
        ...
        ```
    
        What might save us is to somehow in Transaction.extension returns a
        dict-subclass object that is somehow pickled to the exact bytes remembered when
        it was created. However, after briefly checking, I could not find a mechanism
        to do so yet...
    
    @jerome wrote (!19 (comment 129479))
    
        @kirr we already have pytest fixtures to test differently depending on whether
        the ZODB version has support for extension_bytes, so what about using it in the
        test and testing restoring the extension bytes version of the dump only for
        ZODB5 ?
    
    @kirr wrote (!19 (comment 129482))
    
        @jerome, yes we have this, but I believe we should actually fix zodbrestore to
        be reliable whatever ZODB is used. For ZODB5 it works. For ZODB4-wc2 we can
        adjust ZODB code to use extension_bytes similarly to how ZODB5 does. But
        unpatched ZODB4 is currently out of luck. As it was decided that Nexedi will
        use both ZODB4 and ZODB4-wc2, I think we should fix zodbrestore to work on all
        those versions to be reliable.
    
        /cc @tomo
    
    @kirr:
    
    -> No universal ZODB4 fix for now (this would require to monkey patch ZODB in
    several places), so mark "restore with extension" test as xfail similarly to
    how we already do for "dump with extension" test.
    
    This brings -ZODB4 and -ZODB4-wc2 tests back to PASS state.
    
    Even though on ZODB4 extension is restored not bit-to-bit exactly, it is
    restored to be the same dictionary equal to what was used to produce the
    dump. Not ideal, but still not loosing the information in practice.
    
    One more reason to switch to ZODB5...
    aa7e1966
Name
Last commit
Last update
zodbtools Loading commit data...
.gitignore Loading commit data...
.nxdtest Loading commit data...
CHANGELOG.rst Loading commit data...
COPYING Loading commit data...
LICENSE-ZPL.txt Loading commit data...
MANIFEST.in Loading commit data...
README.rst Loading commit data...
setup.py Loading commit data...
tox.ini Loading commit data...