• Kirill Smelkov's avatar
    bigfile/zodb: Do not waste DB space storing trailing zeros for a ZBlk · f696ce92
    Kirill Smelkov authored
    A lot of times data in blocks come shorter than block size and the rest
    of the memory page is zeros (because it was pre-filled zeros by OS when
    page was allocated).
    
    Do a simple heuristic and trim those trailing zeros and not store them
    into DB.
    
    With this change size of DB created by running bigfile and bigarray
    tests changes as following:
    
                bigfile     bigarray
    
        old     145M         35M
        new      24K          6M
    
    Trimming trailing zeros is currently done with str.rstrip('\0') which
    creates a copy. When/if needed this could be optimized to work in-place.
    f696ce92
file_zodb.py 19.2 KB