- 08 Jul, 2019 10 commits
-
-
Kirill Smelkov authored
* master: bigfile: Note that ramh_alloc_page does not add the page to any lists bigfile: Unstub ram_close bigfile/test/test_ram: Don't forget to free allocated Page structs bigfile: RAM must be explicitly free'ed after close bigfile/test: Don't forget to close opened RAM tests: TSAN no longer fails on test_virtmem tox: Don't duplicate setup.py on which for-tests dependencies we need 3rdparty/ccan: Update
-
Kirill Smelkov authored
It was already said there that allocated page is not associated with fileh. However the code is doing more - it does not add the page to ram->lru_list and (obviously) to fileh->dirty_pages lists. Document that explicitly.
-
Kirill Smelkov authored
It should release resources associated with RAM. Make it cal .ram_close from RAM ops. Add corresponding .ram_close to ram_shmfs. This fixes SHMFS_RAM->prefix leak.
-
Kirill Smelkov authored
test_ram is low-level test that tests RAM pages allocation/mmapping. As allocated pages are not integrated with virtmem (not added to any file mapping and RAM->lru_list) the Page structs have to be explicitly freed. Fixes e.g. Direct leak of 80 byte(s) in 1 object(s) allocated from: #0 0x7ff29af46518 in calloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xe9518) #1 0x56131dc22289 in zalloc include/wendelin/utils.h:67 #2 0x56131dc225d6 in ramh_alloc_page bigfile/tests/../ram.c:41 #3 0x56131dc2a19e in main bigfile/tests/test_ram.c:130 #4 0x7ff29ac9f09a in __libc_start_main ../csu/libc-start.c:308
-
Kirill Smelkov authored
Else on-heap allocated RAM object is leaked. Fixes e.g. the following error on ASAN: Direct leak of 56 byte(s) in 1 object(s) allocated from: #0 0x7fc9ef390518 in calloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xe9518) #1 0x555ca792f309 in zalloc include/wendelin/utils.h:67 #2 0x555ca7935f9a in ram_limited_new bigfile/tests/../../t/t_utils.c:35 #3 0x555ca793a0ba in test_file_access_synthetic bigfile/tests/test_virtmem.c:292 #4 0x555ca7967bc4 in main bigfile/tests/test_virtmem.c:1121 #5 0x7fc9ef0e909a in __libc_start_main ../csu/libc-start.c:308
-
Kirill Smelkov authored
Only one place that was using ram_new was missing to call ram_close in the end.
-
Kirill Smelkov authored
For failing case compiler-rt support was added in 2014 - 5 years ago (see links in removed code).
-
Kirill Smelkov authored
-> Use .[test] to refer to them. https://stackoverflow.com/a/41398850/9456786
-
Kirill Smelkov authored
Just an update to latest CCAN - there is actually no changes to modules that we use (tap, array_size, minmax, bitmap, build_assert)
-
Kirill Smelkov authored
-
- 07 Jul, 2019 2 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
- 05 Jul, 2019 9 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
- 04 Jul, 2019 16 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
- 02 Jul, 2019 3 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
... and is still stuck in kernel waiting for FUSE request it sent to self (mkrevfile) to complete. For example: ================== WARNING: DATA RACE Write at 0x00c00009a2a0 by goroutine 17: github.com/hanwen/go-fuse/fuse/nodefs.(*portableHandleMap).Register() /home/kirr/src/neo/src/github.com/hanwen/go-fuse/fuse/nodefs/handle.go:87 +0x130 github.com/hanwen/go-fuse/fuse/nodefs.(*FileSystemConnector).lookupUpdate() /home/kirr/src/neo/src/github.com/hanwen/go-fuse/fuse/nodefs/fsconnector.go:126 +0x65 github.com/hanwen/go-fuse/fuse/nodefs.(*rawBridge).Lookup() /home/kirr/src/neo/src/github.com/hanwen/go-fuse/fuse/nodefs/fsops.go:119 +0x203 github.com/hanwen/go-fuse/fuse.doLookup() /home/kirr/src/neo/src/github.com/hanwen/go-fuse/fuse/opcode.go:335 +0x16e github.com/hanwen/go-fuse/fuse.(*Server).handleRequest() /home/kirr/src/neo/src/github.com/hanwen/go-fuse/fuse/server.go:470 +0x868 github.com/hanwen/go-fuse/fuse.(*Server).loop() /home/kirr/src/neo/src/github.com/hanwen/go-fuse/fuse/server.go:437 +0x1de Previous read at 0x00c00009a2a0 by goroutine 44: main.(*Root).mkrevfile() /home/kirr/src/neo/src/github.com/hanwen/go-fuse/fuse/nodefs/inode.go:66 +0x476 main.(*BigFile).invalidateBlk.func1() /home/kirr/src/wendelin/wendelin.core/wcfs/wcfs.go:980 +0x1b0 main.(*BigFile).invalidateBlk() /home/kirr/src/wendelin/wendelin.core/wcfs/wcfs.go:990 +0x33f main.(*Root).handleδZ.func3() /home/kirr/src/wendelin/wendelin.core/wcfs/wcfs.go:811 +0x8a golang.org/x/sync/errgroup.(*Group).Go.func1() /home/kirr/src/tools/go/g.env/src/golang.org/x/sync/errgroup/errgroup.go:57 +0x64 Goroutine 17 (running) created at: github.com/hanwen/go-fuse/fuse.(*Server).readRequest() /home/kirr/src/neo/src/github.com/hanwen/go-fuse/fuse/server.go:304 +0x5d3 github.com/hanwen/go-fuse/fuse.(*Server).loop() /home/kirr/src/neo/src/github.com/hanwen/go-fuse/fuse/server.go:415 +0x78 github.com/hanwen/go-fuse/fuse.(*Server).Serve() /home/kirr/src/neo/src/github.com/hanwen/go-fuse/fuse/server.go:360 +0x7d main._main.func2() /home/kirr/src/wendelin/wendelin.core/wcfs/wcfs.go:2292 +0x52 Goroutine 44 (running) created at: golang.org/x/sync/errgroup.(*Group).Go() /home/kirr/src/tools/go/g.env/src/golang.org/x/sync/errgroup/errgroup.go:54 +0x73 main.(*Root).handleδZ() /home/kirr/src/wendelin/wendelin.core/wcfs/wcfs.go:810 +0x47d main.(*Root).zwatcher() /home/kirr/src/wendelin/wendelin.core/wcfs/wcfs.go:731 +0x32b main._main() /home/kirr/src/wendelin/wendelin.core/wcfs/wcfs.go:2303 +0x21d1 main.main() /home/kirr/src/wendelin/wendelin.core/wcfs/wcfs.go:2114 +0x89 ================== C: test timed out after 7.0s -> aborting wcfs fuse connection to unblock Traceback (most recent call last): File "/home/kirr/src/tools/go/pygolang/golang/__init__.py", line 122, in _ return f(*argv, **kw) File "/home/kirr/src/wendelin/wendelin.core/wcfs/wcfs_test.py", line 289, in close tf.close() File "/home/kirr/src/wendelin/wendelin.core/wcfs/wcfs_test.py", line 505, in close t.f.close() IOError: [Errno 107] Transport endpoint is not connected Traceback (most recent call last): File "/home/kirr/src/tools/go/pygolang/golang/__init__.py", line 122, in _ return f(*argv, **kw) File "/home/kirr/src/wendelin/wendelin.core/wcfs/wcfs_test.py", line 1284, in test_wcfs_basic at2 = t.commit(zf, {2:'c2', 3:'d2', 5:'f2'}) File "/home/kirr/src/wendelin/wendelin.core/wcfs/wcfs_test.py", line 376, in commit t._wcsync() File "/home/kirr/src/wendelin/wendelin.core/wcfs/wcfs_test.py", line 382, in _wcsync l = t._wc_zheadfh.readline() IOError: [Errno 103] Software caused connection abort before the patch after data race reporting on wcfs.go side and wcfs.go aborting (we set GORACE="halt_on_error") the test was still stuck and one wcfs thread was still alive: ├─py.test,58 /home/kirr/src/wendelin/venv/z-dev/bin/py.test -p no:cacheprovider -vsx -k test_wcfs_basic │ └─(wcfs,67) │ └─{wcfs},82 with 67 being already killed - zombie - and 82 received the same signal but being alive waiting in kernel for answer to a FUSE request: # cat /proc/82/stack [<0>] request_wait_answer+0x206/0x23d [<0>] __fuse_request_send+0x5d/0x80 [<0>] fuse_simple_request+0xc8/0x180 [<0>] fuse_lookup_name+0xfa/0x200 [<0>] fuse_lookup+0x4a/0x110 [<0>] fuse_atomic_open+0xa8/0x100 [<0>] path_openat+0xbe2/0x12a0 [<0>] do_filp_open+0x8e/0x100 [<0>] do_sys_open+0x181/0x220 [<0>] do_syscall_64+0x59/0x35b [<0>] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [<0>] 0xffffffffffffffff and manual `kill -9 82` not making any effect, because after git.kernel.org/linus/a131de0a482a request_wait_answer even after, receiving fatal signal, waits for request to be answered. It was explicitly said in that kernel commit that the way to unblock such deadlocks is /sys/fs/fuse/connection/<X>/abort - so we do it this way on timeout to make sure a test is never stuck forever.
-
Kirill Smelkov authored
-