- 11 Dec, 2020 1 commit
-
-
Kirill Smelkov authored
- it is more logical - this way !wcfs tests don't try to excerceise wcfs functionality (and unexpectedly spawn wcfs server).
-
- 17 Nov, 2020 3 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
Since ZBigFile keeps references to fileh objects that are created through it it forms a file <=> fileh cycle that is not collected without cyclic GC: https://lab.nexedi.com/nexedi/wendelin.core/blob/v0.13-52-ga702d41/bigfile/file_zodb.py#L497 https://lab.nexedi.com/nexedi/wendelin.core/blob/v0.13-52-ga702d41/bigfile/file_zodb.py#L566-571 We did not noticed this leak until now because it is small, but with upcoming wendelin.core 2 it is important to release a fileh, because there is WCFS connection associated with fileh, and if fileh is not released, that connection also stays alive, keeping on-WCFS resources still being used, and preventing WCFS from being unmounted cleanly. -> Add cyclic GC support to PyBigFile / PyBigFileH NOTE: we still don't allow PyVMA <=> PyBigFileH cycles to be collected, because fileh_close called from fileh.__del__ asserts that there are no live mappings left. See added comments for details. There is no known practical need to use such cycles, so this should be ok. See also other patches on cyclic GC topic: - 450ad804 (bigarray: ArrayRef support for BigArray) // adds cyclic GC support for PyVMA - d97641d2 (bigfile/py: Properly untrack PyVMA from GC before dealloc) /proposed-for-review-on nexedi/wendelin.core!12
-
Kirill Smelkov authored
The logic in pyvma_traverse and pyvma_clear needs to be synchronized with PyVMA deallocation. In the next patche we'll be amending this logic, and it will help a reader to keep all those functions together. For the reference: PyVMA support for cyclic GC was introduced in 450ad804 (bigarray: ArrayRef support for BigArray). See also d97641d2 (bigfile/py: Properly untrack PyVMA from GC before dealloc). /proposed-for-review-on nexedi/wendelin.core!12
-
- 09 Nov, 2020 3 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
- 08 Nov, 2020 1 commit
-
-
Kirill Smelkov authored
Since ZBigFile keeps references to fileh objects that are created through it it forms a file <=> fileh cycle that is not collected without cyclic GC: https://lab.nexedi.com/nexedi/wendelin.core/blob/v0.13-52-ga702d41/bigfile/file_zodb.py#L497 https://lab.nexedi.com/nexedi/wendelin.core/blob/v0.13-52-ga702d41/bigfile/file_zodb.py#L566-571 We did not noticed this leak until now because it is small, but with upcoming wendelin.core 2 it is important to release a fileh, because there is WCFS connection associated with fileh, and if fileh is not released, that connection also stays alive, keeping on-WCFS resources still being used, and preventing WCFS from being unmounted cleanly. -> Add cyclic GC support to PyBigFile / PyBigFileH NOTE: we still don't allow PyVMA <=> PyBigFileH cycles to be collected, because fileh_close called from fileh.__del__ asserts that there are no live mappings left. See added comments for details. There is no known practical need to use such cycles, so this should be ok. See also other patches on cyclic GC topic: - 450ad804 (bigarray: ArrayRef support for BigArray) // adds cyclic GC support for PyVMA - d97641d2 (bigfile/py: Properly untrack PyVMA from GC before dealloc)
-
- 05 Nov, 2020 2 commits
-
-
Kirill Smelkov authored
X bigfile/_file_zodb: Fix ZSync to close not only wconn, but also wconn.wc through which wconn was created pywconnOf, before creating wconn, performs wc=wcfs.join(zurl) which creates new filesystem-level connection to WCFS server. This wc is used only to create wconn. So if we do not close wc, after releaseing wconn, it will leak opened file descriptor, to e.g. .wcfs/zurl and prevent tests from finishing cleanly.
-
Kirill Smelkov authored
-
- 04 Nov, 2020 1 commit
-
-
Kirill Smelkov authored
Don't use regular mutex to protect _zsyncReg updates as this can deadlock because one of _zsyncReg mutators (on_zconn_dealloc) is invoked by automatic GC that can be triggered any time.
-
- 03 Nov, 2020 3 commits
-
-
Kirill Smelkov authored
The logic inside ZSync was correct, but it was incorrect to attach zsync to zconn to stay alive and react when that zconn is garbage collected: zsync._on_zconn_dealloc was not called because zsync itself was garbage collected too. This fixes many failures where wconn and associated pinner was not released even though ZODB DB was correctly closed.
-
Kirill Smelkov authored
Excercise the logic that keeps wconn <-> zconn in sync.
-
Kirill Smelkov authored
Else, when runing tests intree `import wcfs` and `import wendelin.wcfs` will give two different modules, and inspecting e.g. wendelin.wcfs at teardown will see fresh module state (_wcregistry) because it was wcfs which was used. Also just `import wcfs` will raise ImportError when run out of tree.
-
- 11 Oct, 2020 1 commit
-
-
Kirill Smelkov authored
Previously if an assert or something failed in spawned thread, the main thread was usually spinning indefinitely = tests hang. -> Switch all threading places to use sync.WorkGroup and this way if a thread fails, all other threads are canceled and the exception is reported back to wg.wait in main thread.
-
- 10 Sep, 2020 1 commit
-
-
Kirill Smelkov authored
-
- 17 Apr, 2020 2 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
Pygolang egg provides "golang" python package, not "pygolang". Bug introduced in 5c8340d2 (*: Use defer for dbclose & friends).
-
- 16 Apr, 2020 1 commit
-
-
Kirill Smelkov authored
-
- 15 Apr, 2020 12 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
- mention in comments that _ZBigFileH not only proxies changes from virtmem -> ZODB, but also the other way: virtmem <- ZODB. - refresh comments, fix typo.
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
It is valid to compare a Page and a VMA only if they belong to the same fileh.
-
Kirill Smelkov authored
-> into vma_page_infilerange(). We will soon need to use this functionality from several places.
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
Start preparing vma early, not after the call to mem_valloc. This codeflow will be more convenient when we add mmap-through-wcfs codepath.
-
- 14 Apr, 2020 9 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-
Kirill Smelkov authored
We are soon going to use this functionality from several places. The place to perform the mmap is changed slightly because vma_mmap_page deduces prot from page->state and for that we have to finish preparing page->state first.
-
Kirill Smelkov authored
-
Kirill Smelkov authored
Freed Page structs must be in PAGE_EMPTY state.
-
Kirill Smelkov authored
-
Kirill Smelkov authored
The code sequence of list_del(&page->lru); bzero(page, sizeof(*page)); /* just in case */ free(page); was open-coded several times.
-
Kirill Smelkov authored
-
Kirill Smelkov authored
-