-
Kirill Smelkov authored
Starting with upcoming ZODB 5.5.2 ZODB tries to preserve `extension_bytes` transaction metadata property in the raw form as it was stored on disk in the database: https://github.com/zopefoundation/ZODB/commit/2f8cc67a However now when running test/gen_testdata.py with ZODB with that patch (and gen_testdata.py refuses to work if it detects that ZODB does not properly supports .extension_bytes property because we want it to be present in the generated test database [1,2]) it now breaks: $ ./gen_testdata.py Traceback (most recent call last): File "./gen_testdata.py", line 230, in <module> main() File "./gen_testdata.py", line 224, in main gen_testdb("%s.fs" % dbname, zext=zext) File "./gen_testdata.py", line 194, in gen_testdb stor.tpc_begin(txn) File "/home/kirr/src/wendelin/z/ZODB/src/ZODB/BaseStorage.py", line 193, in tpc_begin ext = transaction.extension_bytes AttributeError: 'Transaction' object has no attribute 'extension_bytes' The breakage is because, as specified in ZODB interfaces[3,4], storage requires ZODB.IStorageTransactionMetaData, not transaction.ITransaction instance gen_testdata.py was using. The script used to work before just by luck. The fix is to convert transaction instance into storage transaction metadata object for the place where we talk to storage at raw level. HOWEVER, when checking regenerated database and its dump I noticed: ZODB >= 5.4.0 uses pickle protocol 3 on both python2 and python3 https://github.com/zopefoundation/ZODB/commit/12ee41c4 In other words it saves e.g. OID of an object as pickle binary, which decodes as bytes on py3 and zodbpickle.binary on py2 when decoding via zodbpickle. However it will result in *DecodeError* when decoding on py2 with standard pickle module. The latter means that ZODB3 will _fail_ to load data from test database, because ZODB3 - contrary to ZODB4 and ZODB5 - uses std pickle module, not zodbpickle. We still care about ZODB3 and in particular it is included into zodbtools test matrix: https://lab.nexedi.com/nexedi/zodbtools/blob/7bc0385e/tox.ini#L9-14 so we cannot break it. -> Temporarily patch ZODB at runtime to make sure it emits data with older protocol and without using zodbpickle.binary for oid, so that generated test database could be loaded on ZODB3 as well. gen_testdata.py now works with latest ZODB, but produces exactly the same bit-to-bit output as before. [1] https://lab.nexedi.com/nexedi/zodbtools/blob/7bc0385e/zodbtools/test/gen_testdata.py#L215-217 [2] https://lab.nexedi.com/nexedi/zodbtools/blob/7bc0385e/zodbtools/test/testutil.py#L31-63 [3] https://github.com/zopefoundation/ZODB/blob/5.5.1-35-gb5895a5c2/src/ZODB/interfaces.py#L815-L818 [4] https://github.com/zopefoundation/ZODB/blob/5.5.1-35-gb5895a5c2/src/ZODB/interfaces.py#L538-L575 /reviewed-on nexedi/zodbtools!15
0b6f99da