• Kirill Smelkov's avatar
    go/zodb/fs1: Fix access to whiteout · 543041a3
    Kirill Smelkov authored
    A data record with len(data)=0 and backpointer=0 is considered by
    FileStorage/py as "no data":
    
    https://github.com/zopefoundation/ZODB/blob/5.6.0-15-g22d1405d4/src/ZODB/FileStorage/FileStorage.py#L576-L582
    
    Even though currently it is not possible to create such data record via
    FileStorage(py).deleteObject (becase it raises POSKeyError if there is
    no previous object revision), being able to use such data records is
    semantically useful in overlayed DemoStorage settings, where δ part
    marks an object that exists only in base with delete record whiteout.
    
    It is also generally meaningfull to be able to create "delete" record
    even if object was not previously existing: "deleteObject" is actually
    similar to "store" (and so should be better named as "storeDelete"). If
    one wants to store deletion, there should not be a reason to reject it,
    because deleteObject already has seatbelt in the form of oldserial, and
    if the user calls deleteObject(oid, oldserial=z64), he/she is already
    telling that "I know that this object does not exist in this storage
    (oldserial=z64), but still please create a deletion record for it". Once
    again this is useful in overlayed DemoStorage settings described above.
    
    For the reference, such whiteout deletion records pass ZODB/scripts/fstest just fine.
    
    Even though FileStorage/py loads such data records just fine, on
    FileStorage/go side it was not the case - DataHeader.LoadBackRef, even
    with backpointer=0, was verifying that backpointer to be valid and
    failing seeing it might overlap with current transaction:
    
        === RUN   TestLoadWhiteout
        2021/03/17 06:40:58 index load: open testdata/whiteout.fs.index: no such file or directory
        2021/03/17 06:40:58 testdata/whiteout.fs: index rebuild...
            filestorage_test.go:398: load 0000000000000017:0000000000000001: bad err:
                have: testdata/whiteout.fs: load 0000000000000017:0000000000000001: testdata/whiteout.fs: data record @27: check: backpointer (0) overlaps with txn (4)
                want: testdata/whiteout.fs: load 0000000000000017:0000000000000001: 0000000000000001: object was not yet created
    
    It was a thinko: backPos==0 was already kind of handled in LoadBackRef,
    but only in one verification case. -> Fix all checks not to trigger when
    seeing backPos=0. DataHeader.LoadBack - the caller of LoadBackRef -
    already handles returned backPos=0 as "no data".
    543041a3
format.go 23 KB