1. 17 Mar, 2021 9 commits
    • Kirill Smelkov's avatar
      . · d731fd4e
      Kirill Smelkov authored
      d731fd4e
    • Kirill Smelkov's avatar
      Merge branch 'master' into t · f6560f86
      Kirill Smelkov authored
      * master:
        go/zodb/fs1: Fix access to whiteout
        go/zodb/fs1tools: tests: Use fresh dumper for each subtest
        go/zodb/fs1: Fix Python database generator to work with recent zodbtools
        go/zodb/zeo: msgpack.{Encode,Decode} -> msgpack.{Marshal,Unmarshal}
        go/neo/proto: UUID -> NID (NodeID)
        go/neo/internal/xsha1: New package to compute NEO-flavoured SHA1
        go/neo/t/tzodb.go: Switch to tasks and task-integrated logging
        go/internal/task: New package  (draft)
        go/internal/log: New package  (draft)
        go/internal/xcontext/task: New package  (draft)
      f6560f86
    • Kirill Smelkov's avatar
      . · bce76209
      Kirill Smelkov authored
      bce76209
    • Kirill Smelkov's avatar
      go/zodb/fs1: Fix access to whiteout · 543041a3
      Kirill Smelkov authored
      A data record with len(data)=0 and backpointer=0 is considered by
      FileStorage/py as "no data":
      
      https://github.com/zopefoundation/ZODB/blob/5.6.0-15-g22d1405d4/src/ZODB/FileStorage/FileStorage.py#L576-L582
      
      Even though currently it is not possible to create such data record via
      FileStorage(py).deleteObject (becase it raises POSKeyError if there is
      no previous object revision), being able to use such data records is
      semantically useful in overlayed DemoStorage settings, where δ part
      marks an object that exists only in base with delete record whiteout.
      
      It is also generally meaningfull to be able to create "delete" record
      even if object was not previously existing: "deleteObject" is actually
      similar to "store" (and so should be better named as "storeDelete"). If
      one wants to store deletion, there should not be a reason to reject it,
      because deleteObject already has seatbelt in the form of oldserial, and
      if the user calls deleteObject(oid, oldserial=z64), he/she is already
      telling that "I know that this object does not exist in this storage
      (oldserial=z64), but still please create a deletion record for it". Once
      again this is useful in overlayed DemoStorage settings described above.
      
      For the reference, such whiteout deletion records pass ZODB/scripts/fstest just fine.
      
      Even though FileStorage/py loads such data records just fine, on
      FileStorage/go side it was not the case - DataHeader.LoadBackRef, even
      with backpointer=0, was verifying that backpointer to be valid and
      failing seeing it might overlap with current transaction:
      
          === RUN   TestLoadWhiteout
          2021/03/17 06:40:58 index load: open testdata/whiteout.fs.index: no such file or directory
          2021/03/17 06:40:58 testdata/whiteout.fs: index rebuild...
              filestorage_test.go:398: load 0000000000000017:0000000000000001: bad err:
                  have: testdata/whiteout.fs: load 0000000000000017:0000000000000001: testdata/whiteout.fs: data record @27: check: backpointer (0) overlaps with txn (4)
                  want: testdata/whiteout.fs: load 0000000000000017:0000000000000001: 0000000000000001: object was not yet created
      
      It was a thinko: backPos==0 was already kind of handled in LoadBackRef,
      but only in one verification case. -> Fix all checks not to trigger when
      seeing backPos=0. DataHeader.LoadBack - the caller of LoadBackRef -
      already handles returned backPos=0 as "no data".
      543041a3
    • Kirill Smelkov's avatar
      go/zodb/fs1tools: tests: Use fresh dumper for each subtest · a708663d
      Kirill Smelkov authored
      We were reusing Dumper instance in between testDump subtests. This was
      not noticed, as it was only "1" and then "empty" case, because "emtpy"
      has no transactions. However in the next patch we'll add another
      subcase, and if the dumper instance is not reset, it will think that it
      starts from transaction number non-zero, which would differ from fresh
      dumper output.
      
      -> Fix it.
      a708663d
    • Kirill Smelkov's avatar
      go/zodb/fs1: Fix Python database generator to work with recent zodbtools · fc69e00d
      Kirill Smelkov authored
      As Zodbtools dropped ZODB3 support its run_with_zodb3py2_compat was
      renamed to run_with_zodb4py2_compat:
      
      nexedi/zodbtools@c59a54ca
      fc69e00d
    • Kirill Smelkov's avatar
      go/zodb/zeo: msgpack.{Encode,Decode} -> msgpack.{Marshal,Unmarshal} · 363e6e61
      Kirill Smelkov authored
      Encode/decode was deprecated and removed in recent github.com/shamaton/msgpack.
      363e6e61
    • Kirill Smelkov's avatar
      . · bcea2cb6
      Kirill Smelkov authored
      bcea2cb6
    • Kirill Smelkov's avatar
      . · fc1e058b
      Kirill Smelkov authored
      fc1e058b
  2. 16 Mar, 2021 2 commits
  3. 15 Mar, 2021 14 commits
  4. 03 Mar, 2021 1 commit
  5. 02 Mar, 2021 4 commits
  6. 22 Feb, 2021 10 commits