• Kirill Smelkov's avatar
    bigfile/py: Dump pybuf referrers if pybuf->ob_refcnt != 1 before dying in loadblk epilogue · 20b41a5a
    Kirill Smelkov authored
    Instead of only printing "BUG" let's print information about objects
    which still refer to pybuf - to help debugging.
    
    For example with the following artificial pybuf leak
    
    ```
    diff --git a/bigfile/tests/test_basic.py b/bigfile/tests/test_basic.py
    index c737621..f5e057a 100644
    --- a/bigfile/tests/test_basic.py
    +++ b/bigfile/tests/test_basic.py
    @@ -126,6 +126,7 @@ def test_basic():
    
     # test that python exception state is preserved across pagefaulting
     def test_pagefault_savestate():
    +    zzz = []
         class BadFile(BigFile):
             def loadblk(self, blk, buf):
                 # simulate some errors in-between to overwrite thread exception
    @@ -154,6 +155,7 @@ def loadblk(self, blk, buf):
                 # which result in holding additional ref to buf, but loadblk caller
                 # will detect and handle this situation via garbage-collecting
                 # above cycle.
    +            zzz.append(buf)
    
                 self.loadblk_run = 1
    ```
    
    it dies this way:
    
        bigfile/_bigfile.c:567 pybigfile_loadblk WARN: pybuf->ob_refcnt != 1 even after GC:
        pybuf (ob_refcnt=2):    <read-write buffer ptr 0x7f08d3e88000, size 2097152 at 0x7f08d48b7070>
        pybuf referrers:        [[<read-write buffer ptr 0x7f08d3e88000, size 2097152 at 0x7f08d48b7070>]]
        bigfile/_bigfile.c:573 pybigfile_loadblk        BUG!
    20b41a5a
_bigfile.c 28.3 KB