tests: Adjust testdata FileStorage for current Python on the fly
FileStorage/py2 uses `FS21` magic in file header, whereas FileStorage/py3 uses `FS30` magic: https://github.com/zopefoundation/ZODB/blob/0e72b8b13657/src/ZODB/_compat.py#L39 https://github.com/zopefoundation/ZODB/blob/0e72b8b13657/src/ZODB/_compat.py#L74 And if, upon opening the database, file magic does not match to what ZODB expects, open is rejected: https://github.com/zopefoundation/ZODB/blob/0e72b8b13657/src/ZODB/FileStorage/FileStorage.py#L88 https://github.com/zopefoundation/ZODB/blob/0e72b8b13657/src/ZODB/FileStorage/FileStorage.py#L1625-L1630 This is done with the idea for a database, that was written from Python2, to be rejected to be opened from Python3 and vice-versa because strings/bytes semantics changed in between py23. As the result, many zodbtools tests currently fail on py3 when they try to access prepared FileStorage database in testdata, because that database was originally prepared on py2. Here is, for example, how test_zodbdump fails: ___________________________ test_zodbdump[zext-raw] ____________________________ zext = <function zext.<locals>._ at 0x7f28530bf9d0>, pretty = 'raw' @mark.parametrize('pretty', ('raw', 'zpickledis')) def test_zodbdump(zext, pretty): tdir = dirname(__file__) zkind = '_!zext' if zext.disabled else '' > stor = FileStorage('%s/testdata/1%s.fs' % (tdir, zkind), read_only=True) zodbtools/test/test_dump.py:41: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../ZODB/src/ZODB/FileStorage/FileStorage.py:315: in __init__ self._pos, self._oid, tid = read_index( _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ file = <_io.BufferedReader name='/home/kirr/src/wendelin/z/zodbtools/zodbtools/test/testdata/1.fs'> name = '/home/kirr/src/wendelin/z/zodbtools/zodbtools/test/testdata/1.fs' index = <ZODB.fsIndex.fsIndex object at 0x7f2852fee2b0>, tindex = {} stop = b'\xff\xff\xff\xff\xff\xff\xff\xff' ltid = b'\x00\x00\x00\x00\x00\x00\x00\x00', start = 4 maxoid = b'\x00\x00\x00\x00\x00\x00\x00\x00', recover = 0, read_only = True def read_index(file, name, index, tindex, stop=b'\377'*8, ltid=z64, start=4, maxoid=z64, recover=0, read_only=0): """Scan the file storage and update the index.""" ... if file_size: if file_size < start: raise FileStorageFormatError(file.name) seek(0) if read(4) != packed_version: > raise FileStorageFormatError(name) E ZODB.FileStorage.FileStorage.FileStorageFormatError: /home/kirr/src/wendelin/z/zodbtools/zodbtools/test/testdata/1.fs ../ZODB/src/ZODB/FileStorage/FileStorage.py:1630: FileStorageFormatError Since zodbtools primarily work on raw data without decoding stored pickles, unlike Zope or ERP5, it should not be a problem for zodbtools to work on py3 with the database that was prepared on py2. -> Adjust all tests to use FileStorage data generated on the fly based on original files in testdata/ but with FileStorage header being rewritten to match current python.
Showing
Please register or sign in to comment