1. 17 Aug, 2021 3 commits
  2. 16 Aug, 2021 11 commits
  3. 12 Aug, 2021 1 commit
    • Kirill Smelkov's avatar
      X wcfs: tests: Fix tFile.assertBlk not to segfault on a test failure · 7b0c301c
      Kirill Smelkov authored
      See added comments to wcfs_test.py for details on how that can happen.
      Fixes test segmentation faults like
      
          $ WENDELIN_CORE_TEST_DB="<zeo>" python -m pytest -vs -k test_wcfs_watch_vs_access
      
          wcfs_test.py::test_wcfs_watch_vs_access
          ------------------------------- live log setup --------------------------------
          INFO     ZEO.ClientStorage:ClientStorage.py:263 ('localhost', 20106) ClientStorage (pid=36942) created RW/normal for storage: '1'
          INFO     ZEO.cache:cache.py:217 created temporary cache file '<fdopen>'
          INFO     ZEO.ClientStorage:ClientStorage.py:574 ('localhost', 20106) Testing connection <ManagedClientConnection ('127.0.0.1', 20106)>
          INFO     ZEO.zrpc.Connection('C'):connection.py:365 (127.0.0.1:20106) received handshake 'Z4'
          INFO     ZEO.ClientStorage:ClientStorage.py:580 ('localhost', 20106) Server authentication protocol None
          INFO     ZEO.ClientStorage:ClientStorage.py:640 ('localhost', 20106) Connected to storage: ('localhost', 20106)
          INFO     ZEO.ClientStorage:ClientStorage.py:1326 ('localhost', 20106) No verification necessary -- empty cache
          INFO     ZEO.ClientStorage:ClientStorage.py:728 ('localhost', 20106) Disconnected from storage: "('localhost', 20106)"
          -------------------------------- live log call --------------------------------
          INFO     ZEO.ClientStorage:ClientStorage.py:263 ('localhost', 20106) ClientStorage (pid=36942) created RW/normal for storage: '1'
          INFO     ZEO.cache:cache.py:217 created temporary cache file '<fdopen>'
          INFO     ZEO.ClientStorage:ClientStorage.py:574 ('localhost', 20106) Testing connection <ManagedClientConnection ('127.0.0.1', 20106)>
          INFO     ZEO.zrpc.Connection('C'):connection.py:365 (127.0.0.1:20106) received handshake 'Z4'
          INFO     ZEO.ClientStorage:ClientStorage.py:580 ('localhost', 20106) Server authentication protocol None
          INFO     ZEO.ClientStorage:ClientStorage.py:640 ('localhost', 20106) Connected to storage: ('localhost', 20106)
          INFO     ZEO.ClientStorage:ClientStorage.py:1326 ('localhost', 20106) No verification necessary -- empty cache
          INFO     root:__init__.py:294 wcfs: starting for zeo://localhost:20106 ...
          wcfs: 2021/08/13 02:27:40 zodb: FIXME: open zeo://localhost:20106: raw cache is not ready for invalidations -> NoCache forced
          INFO     root:__init__.py:335 wcfs: started pid37431 @ /dev/shm/wcfs/e7630c831aeed36692d06459de5a25a745eb9d76
      
          M: commit -> @at0 (03e2107fabf002ee)
      
          M: commit -> @at1 (03e2107fac097466)
          M:      f<0000000000000002>     [2]
      
          M: commit -> @at2 (03e2107fac3df2aa)
          M:      f<0000000000000002>     [2, 3, 5]
      
          M: commit -> @at3 (03e2107fac5ef011)
          M:      f<0000000000000002>     [2, 5]
      
          C: setup watch f<0000000000000002> @at3 (03e2107fac5ef011)
          #  pinok: {}
      
          C: setup watch f<0000000000000002> @at3 (03e2107fac5ef011)
          #  pinok: {}
      
          C: setup watch f<0000000000000002> @at2 (03e2107fac3df2aa)
          #  pinok: {2: @at2 (03e2107fac3df2aa)}
      
          M: commit -> @at4 (03e2107face33c77)
          M:      f<0000000000000002>     [2, 5, 6]
      
          >>> Change history by file:
      
          f<0000000000000002>:
                                          0 1 2 3 4 5 6 7
                                          a b c d e f g h
                  @at0 (03e2107fabf002ee)
                  @at1 (03e2107fac097466)     2
                  @at2 (03e2107fac3df2aa)     2 3   5
                  @at3 (03e2107fac5ef011)     2     5
                  @at4 (03e2107face33c77)     2     5 6
      
          INFO     ZEO.ClientStorage:ClientStorage.py:728 ('localhost', 20106) Disconnected from storage: "('localhost', 20106)"
          INFO     root:__init__.py:401 wcfs: unmount/stop wcfs pid37431 @ /dev/shm/wcfs/e7630c831aeed36692d06459de5a25a745eb9d76
          WARNING  root:__init__.py:548 fuse_unmount /dev/shm/wcfs/e7630c831aeed36692d06459de5a25a745eb9d76: failed: fusermount: failed to unmount /dev/shm/wcfs/e7630c831aeed36692d06459de5a25a745eb9d76: Device or resource busy
          WARNING  root:__init__.py:533 # lsof /dev/shm/wcfs/e7630c831aeed36692d06459de5a25a745eb9d76
          WARNING  root:__init__.py:541
          WARNING  root:__init__.py:543 (lsof failed)
          WARNING  root:__init__.py:461 -> kill -TERM wcfs.go ...
          WARNING  root:__init__.py:464 -> abort FUSE connection ...
          Segmentation fault: read @00007f6e36bfe000
          /srv/slapgrid/slappart91/srv/runner/software/3335682bae677c2d474f9244e578f64b/parts/wendelin.core/wcfs/client/./../../bigfile/liblibvirtmem.so(dump_traceback+0x1b)[0x7f6f80844e4b]
          /srv/slapgrid/slappart91/srv/runner/software/3335682bae677c2d474f9244e578f64b/parts/wendelin.core/wcfs/client/./../../bigfile/liblibvirtmem.so(+0x3956)[0x7f6f80841956]
          /lib/x86_64-linux-gnu/libpthread.so.0(+0x12730)[0x7f6f83117730]
          /srv/slapgrid/slappart91/srv/runner/software/3335682bae677c2d474f9244e578f64b/parts/wendelin.core/wcfs/internal/wcfs_test.so(+0x10860)[0x7f6e3e2eb860]
          /srv/slapgrid/slappart91/srv//runner//shared/python2.7/93d57ff089fd75f374514794469a0538/bin/python2.7(PyEval_EvalFrameEx+0x7b5)[0x4d2dc5]
          /srv/slapgrid/slappart91/srv//runner//shared/python2.7/93d57ff089fd75f374514794469a0538/bin/python2.7(PyEval_EvalCodeEx+0x2cc)[0x4d1abc]
          /srv/slapgrid/slappart91/srv//runner//shared/python2.7/93d57ff089fd75f374514794469a0538/bin/python2.7[0x51b92e]
          /srv/slapgrid/slappart91/srv/runner/software/3335682bae677c2d474f9244e578f64b/develop-eggs/pygolang-0.0.8-py2.7-linux-x86_64.egg/golang/_golang.so(+0xc8b0)[0x7f6f8182b8b0]
          /srv/slapgrid/slappart91/srv/runner/software/3335682bae677c2d474f9244e578f64b/develop-eggs/pygolang-0.0.8-py2.7-linux-x86_64.egg/golang/_golang.so(+0x14ab4)[0x7f6f81833ab4]
          /srv/slapgrid/slappart91/srv//runner//shared/python2.7/93d57ff089fd75f374514794469a0538/bin/python2.7[0x54bbb4]
          /lib/x86_64-linux-gnu/libpthread.so.0(+0x7fa3)[0x7f6f8310cfa3]
          /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7f6f82eae4cf]
          Segmentation fault (core dumped)
      
      Which looks under gdb as
      
          #0  on_pagefault (sig=<optimized out>, si=0x7f6dde7fb570, _uc=<optimized out>) at bigfile/pagefault.c:171
          #1  <signal handler called>
          #2  __pyx_pf_8wendelin_4wcfs_8internal_9wcfs_test_read_nogil (__pyx_self=<optimized out>, __pyx_v_mem=...) at wcfs/internal/wcfs_test.cpp:3103
          #3  __pyx_pw_8wendelin_4wcfs_8internal_9wcfs_test_1read_nogil (__pyx_self=<optimized out>, __pyx_arg_mem=<optimized out>) at wcfs/internal/wcfs_test.cpp:3029
          #4  0x00000000004d2dc5 in call_function (oparg=<optimized out>, pp_stack=0x7f6dde7fbc88) at Python/ceval.c:4364
          #5  PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:3013
          #6  0x00000000004d1abc in PyEval_EvalCodeEx (co=0x7f6f8094cbb0, globals=<optimized out>, locals=locals@entry=0x0, args=args@entry=0x7f6f82d72068, argcount=<optimized out>, kws=kws@entry=0x7f6f82d72068, kwcount=0, defs=0x0, defcount=0,
              closure=0x7f6e3c6e7110) at Python/ceval.c:3608
          #7  0x000000000051b92e in function_call (func=0x7f6e3c711150, arg=0x7f6f82d72050, kw=0x7f6e3c710b90) at Objects/funcobject.c:523
          #8  0x00007f6f8182b8b0 in __Pyx_PyObject_Call (func=0x7f6e3c711150, arg=<optimized out>, kw=<optimized out>) at golang/_golang.cpp:15660
          #9  0x00007f6f81833ab4 in __pyx_f_6golang_7_golang___goviac (__pyx_v_arg=0x7f6e3c70f5f0) at golang/_golang.cpp:3466
          #10 __pyx_f_6golang_7_golang__goviac (__pyx_v_arg=__pyx_v_arg@entry=0x7f6e3c70f5f0) at golang/_golang.cpp:3350
          #11 0x000000000054bbb4 in pythread_wrapper (arg=<optimized out>) at Python/thread_pthread.h:178
          #12 0x00007f6f8310cfa3 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
          #13 0x00007f6f82eae4cf in clone () from /lib/x86_64-linux-gnu/libc.so.6
      7b0c301c
  4. 11 Aug, 2021 4 commits
  5. 09 Aug, 2021 4 commits
  6. 06 Aug, 2021 6 commits
  7. 05 Aug, 2021 4 commits
    • Kirill Smelkov's avatar
      X test that shows problem of SliceByRootRev where untracked blocks are not... · 4f707117
      Kirill Smelkov authored
      X test that shows problem of SliceByRootRev where untracked blocks are not added uniformly into whole history
      
      === RUN   TestΔFtailSliceXXX
      2021/08/05 18:07:35 zodb: FIXME: open /tmp/TestΔFtailSliceXXX2265944622/001/1.fs: raw cache is not ready for invalidations -> NoCache forced
          δftail_test.go:689: slice (@at0,@at2]:
              have: [@at1·{0 1}S @at2·{1}S]
              want: [@at1·{0 1}S @at2·{0 1}S]
      4f707117
    • Kirill Smelkov's avatar
      . · 8b190e24
      Kirill Smelkov authored
      8b190e24
    • Kirill Smelkov's avatar
      . · e7bc6180
      Kirill Smelkov authored
      e7bc6180
    • Kirill Smelkov's avatar
      . · ec505b13
      Kirill Smelkov authored
      ec505b13
  8. 04 Aug, 2021 5 commits
  9. 03 Aug, 2021 1 commit
  10. 30 Jul, 2021 1 commit
    • Kirill Smelkov's avatar
      X Fix mlock2 build on Debian 8 · c2423296
      Kirill Smelkov authored
      @rporchetto reports build failure on Debian 8 / Linux 3.16
      
          [2021-07-30 15:40:35,677] INFO     gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/srv/slapgrid/slappart19/srv/runner/shared/python2.7/3b7a074d1ded44046871b13502341482/include/python2.7 -c wcfs/internal/mm.c -o build/temp.linux-x86_64-2.7/wcfs/internal/mm.o
          [2021-07-30 15:40:35,677] INFO     wcfs/internal/mm.c: In function 'mlock2':
          [2021-07-30 15:40:35,677] INFO     wcfs/internal/mm.c:618:28: error: 'SYS_mlock2' undeclared (first use in this function); did you mean 'SYS_mlock'?
          [2021-07-30 15:40:35,677] INFO              long err = syscall(SYS_mlock2, addr, len, flags);
          [2021-07-30 15:40:35,677] INFO                                 ^~~~~~~~~~
          [2021-07-30 15:40:35,677] INFO                                 SYS_mlock
      
      Fix the build.
      
      NOTE mlock2 was added in Linux 4.3.
      
      Similarly MCL_ONFAULT is not provided on that old glibc 2.19:
      
          [2021-07-30 15:40:35,677] INFO     wcfs/internal/mm.c:986:55: error: 'MCL_ONFAULT' undeclared here (not in a function); did you mean 'MLOCK_ONFAULT'?
          [2021-07-30 15:40:35,677] INFO        __pyx_e_8wendelin_4wcfs_8internal_2mm_MCL_ONFAULT = MCL_ONFAULT,
          [2021-07-30 15:40:35,678] INFO                                                            ^~~~~~~~~~~
          [2021-07-30 15:40:35,678] INFO                                                            MLOCK_ONFAULT
      
      -> Comment MCL_ONFAULT for now since we do not actually use it anywhere yet.
      c2423296