• Levin Zimmermann's avatar
    wcfs: v↑ go dependencies · 764d4da8
    Levin Zimmermann authored
    This patch updates:
    
    - github.com/golang/glog: we already wanted to do so in
        nexedi/wendelin.core!23,
        but we deferred it to keep go 1.18 support. However in recent patches
        we already dropped go 1.18 support and we can therefore update glog now.
    - lab.nexedi.com/kirr/neo/go: add fix in handshake, see here for more information:
        kirr/neo@d75f4ac2 and
        kirr/neo@03db1d8a
    
    This patch doesn't update:
    
    - github.com/hanwen/go-fuse: This was updated upstream and Kirill already
        reviewed and integrated patches in custom branch. However when updating
        go-fuse to v2.4.3-0.20240904154523-9546fc238dc6 (this is
        kirr/go-fuse@9546fc23),
        WCFS tests fail on my machine [1] => let's defer update
    - github.com/kisielk/og-rek: there are new patches that will be needed
        in the future, but we didn't update NEO/go og-rek dependency yet,
        so let's defer the update in wendelin.core until we updated og-rek
        in NEO/go
    - github.com/johncgriffin/overflow: no update on upstream
    - github.com/pkg/errors: no update on upstream
    - github.com/stretchr/testify: This was already updated with
        nexedi/wendelin.core@c559ec1a
        'testify' seems to have a major release in the future which may break
        some of our test code, but for now major version 1 is still the
        stable release.
    
    ----
    kirr: I confirm that
    kirr/go-fuse@9546fc23 brings in
    regression to WCFS tests. It seems I missed some error in that go-fuse
    update and it will need to be bisected and debugged.
    
    ---
    
    [1] Test failure log:
    
    ========================================== FAILURES ==========================================
    ______________________________________ test_wcfs_basic _______________________________________
    
        @func
        def test_wcfs_basic():
            t = tDB(); zf = t.zfile
            defer(t.close)
    
            # >>> lookup non-BigFile -> must be rejected
            with raises(OSError) as exc:
                t.wc._stat("head/bigfile/%s" % h(t.nonzfile._p_oid))
            assert exc.value.errno == EINVAL
    
            # >>> file initially empty
            f = t.open(zf)
            f.assertCache([])
            f.assertData ([], mtime=t.at0)
    
            # >>> (@at1) commit data -> we can see it on wcfs
            at1 = t.commit(zf, {2:'c1'})
    
            f.assertCache([0,0,0])  # initially not cached
            f.assertData (['','','c1'], mtime=t.head)
    
            # >>> (@at2) commit again -> we can see both latest and snapshotted states
            # NOTE blocks e(4) and f(5) will be accessed only in the end
            at2 = t.commit(zf, {2:'c2', 3:'d2', 5:'f2'})
    
            # f @head
    >       f.assertCache([1,1,0,0,0,0])
    
    wcfs/wcfs_test.py:1341:
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    
    t = <wcfs.wcfs_test.tFile instance at 0x7ff61457b960>, incorev = [1, 1, 0, 0, 0, 0]
    
        def assertCache(t, incorev):
    >       assert t.cached() == incorev
    E       assert [0, 0, 0, 0, 0, 0] == [1, 1, 0, 0, 0, 0]
    E         At index 0 diff: 0 != 1
    E         Use -v to get the full diff
    
    wcfs/wcfs_test.py:791: AssertionError
    ------------------------------------ Captured stdout call ------------------------------------
    
    M: commit -> @at0 (03fb5dfbe3c1cd55)
    
    M: commit -> @at1 (03fb5dfbe4936a66)
    M:      f<0000000000000002>     [2]
    
    M: commit -> @at2 (03fb5dfbe4d01166)
    M:      f<0000000000000002>     [2, 3, 5]
    >>> Change history by file:
    
    f<0000000000000002>:
                                    0 1 2 3 4 5 6 7
                                    a b c d e f g h
            @at0 (03fb5dfbe3c1cd55)
            @at1 (03fb5dfbe4936a66)     2
            @at2 (03fb5dfbe4d01166)     2 3   5
    
    ------------------------------------ Captured stderr call ------------------------------------
    I0917 12:43:53.392222  124283 wcfs.go:2752] start "/dev/shm/wcfs/0ca22ca24e4cff2d01c10aa546fe5d5ac64bce72" "file:///tmp/testdb_fs.z5ZoMH/1.fs"
    I0917 12:43:53.392282  124283 wcfs.go:2758] (built with go1.21.13)
    W0917 12:43:53.392404  124283 storage.go:232] zodb: FIXME: open file:///tmp/testdb_fs.z5ZoMH/1.fs: raw cache is not ready for invalidations -> NoCache forced
    W0917 12:43:53.567807  124283 wcfs.go:2331] /head/bigfile: lookup "0000000000000001": bigfopen 0000000000000001 @03fb5dfbe3c1cd55: invalid argument: ZODB.Broken("persistent.Persistent") is not a ZBigFile
    I0917 12:43:53.710208  124283 wcfs.go:2933] stop "/dev/shm/wcfs/0ca22ca24e4cff2d01c10aa546fe5d5ac64bce72" "file:///tmp/testdb_fs.z5ZoMH/1.fs"
    ------------------------------------- Captured log call --------------------------------------
    WARNING  ZODB.FileStorage:FileStorage.py:412 Ignoring index for /tmp/testdb_fs.z5ZoMH/1.fs
    _________________________________ test_wcfs_watch_vs_access __________________________________
    
        @func
        def test_wcfs_watch_vs_access():
            t = tDB(); zf = t.zfile; at0=t.at0
            defer(t.close)
    
            f = t.open(zf)
            at1 = t.commit(zf, {2:'c1'})
            at2 = t.commit(zf, {2:'c2', 3:'d2', 5:'f2'})
            at3 = t.commit(zf, {0:'a3', 2:'c3', 5:'f3'})
    
            f.assertData(['a3','','c3','d2','x','x'])
            f.assertCache([1,1,1,1,0,0])
    
            # watched + commit -> read -> receive pin messages.
            # read vs pin ordering is checked by assertBlk.
            #
            # f(5) is kept not accessed to check later how wcfs.go handles δFtail
            # rebuild after it sees not yet accessed ZBlk that has change history.
            wl3  = t.openwatch();  w3 = wl3.watch(zf, at3);  assert at3 == t.head
            assert w3.at     == at3
            assert w3.pinned == {}
    
            wl3_ = t.openwatch();  w3_ = wl3_.watch(zf, at3)
            assert w3_.at     == at3
            assert w3_.pinned == {}
    
            wl2  = t.openwatch();  w2 = wl2.watch(zf, at2)
            assert w2.at     == at2
            assert w2.pinned == {0:at0, 2:at2}
    
            # w_assertPin asserts on state of .pinned for {w3,w3_,w2}
            def w_assertPin(pinw3, pinw3_, pinw2):
                assert w3.pinned   == pinw3
                assert w3_.pinned  == pinw3_
                assert w2.pinned   == pinw2
    
            f.assertCache([1,1,1,1,0,0])
            at4 = t.commit(zf, {1:'b4', 2:'c4', 5:'f4', 6:'g4'})
    >       f.assertCache([1,0,0,1,0,0,0])
    
    wcfs/wcfs_test.py:1702:
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    
    t = <wcfs.wcfs_test.tFile instance at 0x7ff614512050>, incorev = [1, 0, 0, 1, 0, 0, ...]
    
        def assertCache(t, incorev):
    >       assert t.cached() == incorev
    E       assert [0, 0, 0, 0, 0, 0, ...] == [1, 0, 0, 1, 0, 0, ...]
    E         At index 0 diff: 0 != 1
    E         Use -v to get the full diff
    
    wcfs/wcfs_test.py:791: AssertionError
    ------------------------------------ Captured stdout call ------------------------------------
    
    M: commit -> @at0 (03fb5dfc0fd82300)
    
    M: commit -> @at1 (03fb5dfc10b92ecc)
    M:      f<0000000000000049>     [2]
    
    M: commit -> @at2 (03fb5dfc10cee9dd)
    M:      f<0000000000000049>     [2, 3, 5]
    
    M: commit -> @at3 (03fb5dfc1100c999)
    M:      f<0000000000000049>     [0, 2, 5]
    
    C: setup watch f<0000000000000049> @at3 (03fb5dfc1100c999)
    
    C: setup watch f<0000000000000049> @at3 (03fb5dfc1100c999)
    
    C: setup watch f<0000000000000049> @at2 (03fb5dfc10cee9dd)
    
    M: commit -> @at4 (03fb5dfc120ed611)
    M:      f<0000000000000049>     [1, 2, 5, 6]
    >>> Change history by file:
    
    f<0000000000000049>:
                                    0 1 2 3 4 5 6 7
                                    a b c d e f g h
            @at0 (03fb5dfc0fd82300)
            @at1 (03fb5dfc10b92ecc)     2
            @at2 (03fb5dfc10cee9dd)     2 3   5
            @at3 (03fb5dfc1100c999) 0   2     5
            @at4 (03fb5dfc120ed611)   1 2     5 6
    
    ------------------------------------ Captured stderr call ------------------------------------
    I0917 12:44:03.733037  125217 wcfs.go:2752] start "/dev/shm/wcfs/0ca22ca24e4cff2d01c10aa546fe5d5ac64bce72" "file:///tmp/testdb_fs.z5ZoMH/1.fs"
    I0917 12:44:03.733126  125217 wcfs.go:2758] (built with go1.21.13)
    W0917 12:44:03.733418  125217 storage.go:232] zodb: FIXME: open file:///tmp/testdb_fs.z5ZoMH/1.fs: raw cache is not ready for invalidations -> NoCache forced
    I0917 12:44:04.475273  125217 wcfs.go:2933] stop "/dev/shm/wcfs/0ca22ca24e4cff2d01c10aa546fe5d5ac64bce72" "file:///tmp/testdb_fs.z5ZoMH/1.fs"
    ============================ 2 failed, 42 passed in 55.81 seconds ============================
    I0917 12:44:17.882140  125540 wcfs.go:2933] stop "/dev/shm/wcfs/c4d833a0bdea4c51decf5425b8ad2cc4d017280f" "file:///tmp/testdb_fs.bvHBy9/1.fs"
    make: *** [Makefile:174: test.wcfs] Error 1
    
    /reviewed-by @kirr
    /reviewed-on nexedi/wendelin.core!30
    764d4da8
go.mod 1.7 KB