1. 02 Nov, 2020 1 commit
    • Kirill Smelkov's avatar
      X wcfs: test: Fix thinko in getting /sys/fs/fuse/connection/<X> for wcfs · 78f36993
      Kirill Smelkov authored
      FUSE puts X as st_dev's minor, which, for minors <= 255 is the same as st_dev.
      However when there are many connections, and minor goes after 255, minor becomes != st_dev:
      
          In [2]: os.makedev(0, 254)
          Out[2]: 254
      
          In [3]: os.makedev(0, 255)
          Out[3]: 255
      
          In [5]: os.makedev(0, 256)
          Out[5]: 1048576
      
      As a result we were constructing wrong patch, and if wcfs was failing we were
      also failing to kill it with something like:
      
          t = <wcfs.wcfs_test.tDB object at 0x7fef78043260>
      
              @func
              def __init__(t):
                  t.root = testdb.dbopen()
                  def _(): # close/unlock db if __init__ fails
                      exc = sys.exc_info()[1]
                      if exc is not None:
                          dbclose(t.root)
                  defer(_)
      
                  assert not os.path.exists(testmntpt)
                  t.wc = wcfs.join(testzurl, autostart=True)
                  assert os.path.exists(testmntpt)
                  assert is_mountpoint(testmntpt)
      
                  # force-unmount wcfs on timeout to unstuck current test and let it fail.
                  # Force-unmount can be done reliably only by writing into
                  # /sys/fs/fuse/connections/<X>/abort. For everything else there are
                  # cases, when wcfs, even after receiving `kill -9`, will be stuck in kernel.
                  # ( git.kernel.org/linus/a131de0a482a makes in-kernel FUSE client to
                  #   still wait for request completion even after fatal signal )
          >       t._wcfuseabort   = open("/sys/fs/fuse/connections/%d/abort" % os.stat(testmntpt).st_dev, "w")
          E       IOError: [Errno 2] No such file or directory: '/sys/fs/fuse/connections/2097264/abort'
      
          wcfs/wcfs_test.py:236: IOError
      
      In the above failure st_dev=2097264 corresponds to X=624:
      
          In [6]: os.minor(2097264)
          Out[6]: 624
      78f36993
  2. 01 Nov, 2020 2 commits
  3. 30 Oct, 2020 5 commits
    • Kirill Smelkov's avatar
      . · 9040bc20
      Kirill Smelkov authored
      9040bc20
    • Kirill Smelkov's avatar
      . · ad860ba6
      Kirill Smelkov authored
      ad860ba6
    • Kirill Smelkov's avatar
      X wcfs: client: Handle fork · 3f83469c
      Kirill Smelkov authored
      Without special care a forked child may interfere in parent-wcfs
      exchange via Python GC -> PyFileH.__del__ -> FileH.close -> message to
      WCFS sent from the child. This actually happens for real when running
      test.py/neo-wcfs because NEO test cluster spawns master and storage
      nodes with just fork without exec.
      
      -> detach from wcfs in child right after fork and deactivate all
      mappings in order not to provide stale data. See top-level comments
      added to wcfs/client/wcfs.cpp for details.
      3f83469c
    • Kirill Smelkov's avatar
      X wcfs: tests: Factor-out waiting for a general condition to become true into waitfor · c2c35851
      Kirill Smelkov authored
      Currently in wcfs_test.py there is only waiting for a proc
      (subprocess.Popen instance) to become ready. However in the next patch
      we'll need to wait via polling for another condition.
      
      -> Generalize the pollwait code into waitfor* variants, and make
      procwait* use waitfor* internally.
      c2c35851
    • Kirill Smelkov's avatar
      X wcfs: client: os: Factor syserr -> string into _sysErrString · 17f98edc
      Kirill Smelkov authored
      Currently the code to convert `int err` or errno into string is usde
      only in _pathError, but in the next patches we'll need it to also handle
      error from pthread_atfork. -> Factor-out to separate function.
      17f98edc
  4. 27 Oct, 2020 1 commit
  5. 25 Oct, 2020 1 commit
  6. 23 Oct, 2020 3 commits
  7. 22 Oct, 2020 5 commits
  8. 21 Oct, 2020 1 commit
  9. 18 Oct, 2020 1 commit
  10. 16 Oct, 2020 6 commits
  11. 15 Oct, 2020 1 commit
  12. 14 Oct, 2020 2 commits
  13. 13 Oct, 2020 2 commits
  14. 12 Oct, 2020 4 commits
  15. 11 Oct, 2020 5 commits
    • Kirill Smelkov's avatar
      . ci · 35392a78
      Kirill Smelkov authored
      35392a78
    • Kirill Smelkov's avatar
      . · ca909d79
      Kirill Smelkov authored
      ca909d79
    • Kirill Smelkov's avatar
      X *: tests: don't hang on exception in non-main thread · 7ecb8f14
      Kirill Smelkov authored
      Previously if an assert or something failed in spawned thread, the main
      thread was usually spinning indefinitely = tests hang. -> Switch all
      threading places to use sync.WorkGroup and this way if a thread fails,
      all other threads are canceled and the exception is reported back to
      wg.wait in main thread.
      7ecb8f14
    • Kirill Smelkov's avatar
      . · a4bfea98
      Kirill Smelkov authored
      a4bfea98
    • Kirill Smelkov's avatar
      . · 212367a2
      Kirill Smelkov authored
      212367a2