1. 24 Jun, 2021 5 commits
    • Kirill Smelkov's avatar
      . · 8402f4c3
      Kirill Smelkov authored
      8402f4c3
    • Kirill Smelkov's avatar
      X rebuild: Serial tests now started to PASS draftly · c34907cf
      Kirill Smelkov authored
          $ go test -v -failfast -run 'TestΔBTail$/rebuild'
          ...
          PASS
          ok      lab.nexedi.com/nexedi/wendelin.core/wcfs/internal/xbtree        176.661s
      c34907cf
    • Kirill Smelkov's avatar
      . · 22d712a5
      Kirill Smelkov authored
      22d712a5
    • Kirill Smelkov's avatar
      X rebuild: Don't return nil for empty ΔPPTreeSubSet - that leads to SIGSEGV · ddb28043
      Kirill Smelkov authored
      --- FAIL: TestΔBTail (0.74s)
          --- FAIL: TestΔBTail/rebuild/T/T1/T-T2/B0:e-B1:f-B2:g→T1/T-T/B0:g-T2/B1:h-B2:g (0.00s)
              --- FAIL: TestΔBTail/rebuild/T/T1/T-T2/B0:e-B1:f-B2:g→T1/T-T/B0:g-T2/B1:h-B2:g/_T{};R (0.00s)
                  --- FAIL: TestΔBTail/rebuild/T/T1/T-T2/B0:e-B1:f-B2:g→T1/T-T/B0:g-T2/B1:h-B2:g/_T{};R/_→T1/T-T/B0:g-T/B1:e (0.00s)
      panic: runtime error: invalid memory address or nil pointer dereference [recovered]
              panic: runtime error: invalid memory address or nil pointer dereference
      [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x632cc1]
      
      goroutine 238 [running]:
      testing.tRunner.func1.2(0x696aa0, 0x893f70)
              /home/kirr/src/tools/go/go/src/testing/testing.go:1143 +0x332
      testing.tRunner.func1(0xc000001b00)
              /home/kirr/src/tools/go/go/src/testing/testing.go:1146 +0x4b6
      panic(0x696aa0, 0x893f70)
              /home/kirr/src/tools/go/go/src/runtime/panic.go:965 +0x1b9
      lab.nexedi.com/nexedi/wendelin.core/wcfs/internal/xbtree.PPTreeSubSet.ApplyΔ(0xc00040f9b0, 0x0)
              /home/kirr/src/neo/src/lab.nexedi.com/nexedi/wendelin.core/wcfs/internal/xbtree/pptreesubset.go:515 +0x41
      lab.nexedi.com/nexedi/wendelin.core/wcfs/internal/xbtree.(*ΔTtail).rebuild(0xc00040f830, 0xc, 0xc00040f500, 0xc000142b40, 0x0, 0x0, 0x0)
              /home/kirr/src/neo/src/lab.nexedi.com/nexedi/wendelin.core/wcfs/internal/xbtree/δbtail.go:371 +0x145
      lab.nexedi.com/nexedi/wendelin.core/wcfs/internal/xbtree.(*ΔBtail).rebuildAll(0xc00015dc00, 0x0, 0x0)
              /home/kirr/src/neo/src/lab.nexedi.com/nexedi/wendelin.core/wcfs/internal/xbtree/δbtail.go:319 +0x18c
      lab.nexedi.com/nexedi/wendelin.core/wcfs/internal/xbtree.xverifyΔBTail_rebuild_TR(0xc000001b00, 0xc000142b40, 0xc00015dc00, 0xc00006a960, 0xc, 0xc0004863c0, 0xc00040f3b0, 0xc00040f5c0, 0xc00040f650, 0xc00040f740, ...)
              /home/kirr/src/neo/src/lab.nexedi.com/nexedi/wendelin.core/wcfs/internal/xbtree/δbtail_test.go:1127 +0x1b6
      lab.nexedi.com/nexedi/wendelin.core/wcfs/internal/xbtree.xverifyΔBTail_rebuild.func1.1.1(0xc000001b00)
              /home/kirr/src/neo/src/lab.nexedi.com/nexedi/wendelin.core/wcfs/internal/xbtree/δbtail_test.go:1074 +0xafa
      testing.tRunner(0xc000001b00, 0xc000555740)
              /home/kirr/src/tools/go/go/src/testing/testing.go:1193 +0xef
      created by testing.(*T).Run
              /home/kirr/src/tools/go/go/src/testing/testing.go:1238 +0x2b3
      exit status 2
      ddb28043
    • Kirill Smelkov's avatar
      X treediff: Fix BUG while computing AB coverage · 9d20f8e8
      Kirill Smelkov authored
      Access to node entries is valid only after the node was activated.
      And it fails otherwise.
      
      --- FAIL: TestΔBTail (0.63s)
          --- FAIL: TestΔBTail/rebuild/T/T1/B0:a-B1:b→T/T1/T-T/B0:c-B1:d (0.00s)
              --- FAIL: TestΔBTail/rebuild/T/T1/B0:a-B1:b→T/T1/T-T/B0:c-B1:d/_T{};R (0.00s)
                  --- FAIL: TestΔBTail/rebuild/T/T1/B0:a-B1:b→T/T1/T-T/B0:c-B1:d/_T{};R/_→T/T/B0:a (0.00s)
      panic: BUG: T000000000000003a points to T000000000000000c as parent in trackSet, but not found in T000000000000000c children [recovered]
              panic: BUG: T000000000000003a points to T000000000000000c as parent in trackSet, but not found in T000000000000000c children
      9d20f8e8
  2. 23 Jun, 2021 11 commits
    • Kirill Smelkov's avatar
      X ΔBtail.Clone had bug that it was aliasing klon and orig data · f60528c9
      Kirill Smelkov authored
      Rebuild tests were "surprisingly" failing when run as part of whole
      testsuite, but succeeding when run separately.
      f60528c9
    • Kirill Smelkov's avatar
      X Fix rebuild with ø @at2 · 450ba707
      Kirill Smelkov authored
      450ba707
    • Kirill Smelkov's avatar
      X Fix bug in PPTreeSubSet.Difference - it was always leaving root node alive · 66bc41ce
      Kirill Smelkov authored
      So that even A \ A was not ø.
      66bc41ce
    • Kirill Smelkov's avatar
      . · 7533257b
      Kirill Smelkov authored
      7533257b
    • Kirill Smelkov's avatar
      . · 6eeaff06
      Kirill Smelkov authored
      6eeaff06
    • Kirill Smelkov's avatar
      X found why TestΔBTailAllStructs was not effective to find δtkeycov bugs · 502e05c2
      Kirill Smelkov authored
      It was due to thinko in iterating through keys set - we were using key
      ordinals instead of keys themselvs. Now it find such bugs:
      
      === RUN   TestΔBTailAllStructs/Update/T1,3/B0:d-B1:a-B3:d,4:g→T1,4/B0:e-B1:a-B4:c
          δbtail_test.go:839: 2: trackSet:
                      have: map[000000000000000c:{pffffffffffffffff c2} 00000000000005c2:{p000000000000000c c0} 0000000000000618:{p000000000000000c c0}]
                      want: map[000000000000000c:{pffffffffffffffff c3} 00000000000005c2:{p000000000000000c c0} 0000000000000618:{p000000000000000c c0} 0000000000000620:{p000000000000000c c0}]
          δbtail_test.go:708: T1,3/B0:d-B1:a-B3:d,4:g -> T1,4/B0:e-B1:a-B4:c  ; tracked={0 1} :
      
              d12:  map[0:{d e} 3:{d ø} 4:{g c}]
              δTok: map[0:{d e} 3:{d ø} 4:{g c}]
              δT:   map[0:{d e}]
      
              δZ:               {000000000000000c 0000000000000618 0000000000000620}
              Tracked^δZ:       {0 1}
              kadj[Tracked^δZ]: {0 1 3 4 9223372036854775807}
              kadj: map[0:{0} 1:{1 3 4 9223372036854775807} 3:{1 3 4 9223372036854775807} 4:{1 3 4 9223372036854775807} 9223372036854775807:{1 3 4 9223372036854775807}]
      
              δtkeycov wrong:
              have: {}
              want: {[3,∞)}
              δT    ∌  δTok[3]
              δT    ∌  δTok[4]
      502e05c2
    • Kirill Smelkov's avatar
      . · 35bbf9b9
      Kirill Smelkov authored
      35bbf9b9
    • Kirill Smelkov's avatar
      . · 7a7c9756
      Kirill Smelkov authored
      7a7c9756
    • Kirill Smelkov's avatar
      X wcfs: assert that keycov only grow · 0917380e
      Kirill Smelkov authored
      Currently fails with e.g.:
      
      (neo) (z-dev) (g.env) kirr@deca:~/src/neo/src/lab.nexedi.com/nexedi/wendelin.core/wcfs/internal/xbtree$ go test -short -v -failfast -run 'TestΔBTail$/Update/T3/B1:a-B8:c→T7/B1:a-B8:c'
      === RUN   TestΔBTail
      2021/06/23 14:08:45 zodb: FIXME: open /tmp/δBTail926507346/1.fs: raw cache is not ready for invalidations -> NoCache forced
      === RUN   TestΔBTail/Update/T3/B1:a-B8:c→T7/B1:a-B8:c
          δbtail_test.go:812: BUG: tkeyCov1 ⊄ tkeyCov2:
                      tkeyCov1: {[3,∞)}
                      tkeyCov2: {[7,∞)}
                      tkeyCov1 \ tkeyCov2: {[3,7)}
          δbtail_test.go:708: T3/B1:a-B8:c -> T7/B1:a-B8:c  ; tracked={8} :
      
              d12:  map[]
              δTok: map[]
              δT:   map[]
      
              δZ:               {000000000000000c}
              Tracked^δZ:       {}
              kadj[Tracked^δZ]: {}
              kadj: map[1:{1 8 9223372036854775807} 8:{1 8 9223372036854775807} 9223372036854775807:{1 8 9223372036854775807}]
      
      --- FAIL: TestΔBTail (0.84s)
          --- FAIL: TestΔBTail/Update/T3/B1:a-B8:c→T7/B1:a-B8:c (0.00s)
      0917380e
    • Kirill Smelkov's avatar
      . · 595afd6a
      Kirill Smelkov authored
      595afd6a
    • Kirill Smelkov's avatar
      X another bug in δtkeyconv computation · f30826a6
      Kirill Smelkov authored
      === RUN   TestΔBTail/Update/T7/B1:a,2:b-B8:c,9:d→T3/T-T/B1:a,2:b-B8:c,9:d
          δbtail_test.go:832: 2: trackSet:
                      have: map[000000000000000c:{pffffffffffffffff c2} 0000000000000041:{p0000000000000043 c0} 0000000000000042:{p0000000000000044 c0} 0000000000000043:{p000000000000000c c1} 0000000000000044:{p000000000000000c c1}]
                      want: map[000000000000000c:{pffffffffffffffff c1} 0000000000000041:{p0000000000000043 c0} 0000000000000043:{p000000000000000c c1}]
          δbtail_test.go:708: T7/B1:a,2:b-B8:c,9:d -> T3/T-T/B1:a,2:b-B8:c,9:d  ; tracked={1} :
      
              d12:  map[]
              δTok: map[]
              δT:   map[]
      
              δZ:               {000000000000000c 0000000000000043 0000000000000044}
              Tracked^δZ:       {}
              kadj[Tracked^δZ]: {}
              kadj: map[1:{1 2 8 9 9223372036854775807} 2:{1 2 8 9 9223372036854775807} 8:{1 2 8 9 9223372036854775807} 9:{1 2 8 9 9223372036854775807} 9223372036854775807:{1 2 8 9 9223372036854775807}]
      
              δtkeycov wrong:
              have: {[7,∞)}
              want: {}
      f30826a6
  3. 22 Jun, 2021 5 commits
  4. 21 Jun, 2021 15 commits
    • Kirill Smelkov's avatar
      . · 7e7f0f94
      Kirill Smelkov authored
      7e7f0f94
    • Kirill Smelkov's avatar
      . · f428df95
      Kirill Smelkov authored
      f428df95
    • Kirill Smelkov's avatar
      . · c13a7bc0
      Kirill Smelkov authored
      c13a7bc0
    • Kirill Smelkov's avatar
      . · d741c8c7
      Kirill Smelkov authored
      d741c8c7
    • Kirill Smelkov's avatar
      . · e09b503e
      Kirill Smelkov authored
      e09b503e
    • Kirill Smelkov's avatar
      . · 2f9a46eb
      Kirill Smelkov authored
      2f9a46eb
    • Kirill Smelkov's avatar
      . · 69449694
      Kirill Smelkov authored
      69449694
    • Kirill Smelkov's avatar
      . · 8eeef71a
      Kirill Smelkov authored
      8eeef71a
    • Kirill Smelkov's avatar
      . · 7d2e4509
      Kirill Smelkov authored
      7d2e4509
    • Kirill Smelkov's avatar
      . · e8d30e1d
      Kirill Smelkov authored
      e8d30e1d
    • Kirill Smelkov's avatar
      . · 6c0c6755
      Kirill Smelkov authored
      6c0c6755
    • Kirill Smelkov's avatar
      . · 364c2727
      Kirill Smelkov authored
      364c2727
    • Kirill Smelkov's avatar
      . · 09489db6
      Kirill Smelkov authored
      09489db6
    • Kirill Smelkov's avatar
      . · cb2fae27
      Kirill Smelkov authored
      cb2fae27
    • Kirill Smelkov's avatar
      Merge branch 't' into t2 · b221b0b6
      Kirill Smelkov authored
      * t:
        .
        X wcfs: Fix ZSync to close wconn on zdb.close, even if zconn stays alive
        X lib/zodb: Connection += onShutdownCallback
        .
        X wcfs: lsof +D misbehaves - don't use it
        X wcfs: _fuse_unmount: Try first `kill -TERM` before `kill -QUIT` wcfs
        X wcfs: Tune _fuse_unmount to include `fusermount -u` error message into raised exception
        X wcfs: Teach start to start successfully even after unclean wcfs shutdown
        fixup! X wcfs: Run fusermount and friends with /bin:/usr/bin always on path
        X wcfs: Run fusermount and friends with /bin:/usr/bin always on path
        X wcfs: Add start to spawn a Server that can be later stopped  (draft)
      b221b0b6
  5. 20 Jun, 2021 1 commit
  6. 18 Jun, 2021 3 commits
    • Kirill Smelkov's avatar
      X wcfs: Fix ZSync to close wconn on zdb.close, even if zconn stays alive · 7203d7ab
      Kirill Smelkov authored
      Even if ZODB DB is closed and releases zconn from its .pool, zconn can
      still stay alive being referenced from any other live python object -
      e.g. from a frame, a traceback etc. This situation in particular happens
      under ERP5's runUnitTest.
      
      As the result, if we don't hook into DB.close and close wconn only on
      zconn GC, files opened on WCFS filesystem stay opened, and WCFS server
      cannot be cleanly unmounted when test run completes.
      
      -> Fix it.
      
      P.S. ZODB@bbd03b3a is good, but does
      not really solve this problem because, once again, zconn stays
      referenced from objects besides DB.
      7203d7ab
    • Kirill Smelkov's avatar
      X lib/zodb: Connection += onShutdownCallback · a26d9659
      Kirill Smelkov authored
      In the next patch we'll need this functionality to subscribe ZSync into
      db.close to know when ZODB Connection is shut down even if it stays
      alive referenced by some other objects.
      a26d9659
    • Kirill Smelkov's avatar
      . · 8dcaa5de
      Kirill Smelkov authored
      8dcaa5de