• Guido van Rossum's avatar
    Patch by Charles G Waldman to avoid a sneaky memory leak in · b8a9659a
    Guido van Rossum authored
    _PyTuple_Resize().  In addition, a change suggested by Jeremy Hylton
    to limit the size of the free lists is also merged into this patch.
    
    Charles wrote initially:
    
    """
    Test Case:  run the following code:
    
    class Nothing:
        def __len__(self):
            return 5
        def __getitem__(self, i):
            if i < 3:
                return i
            else:
                raise IndexError, i
    
    def g(a,*b,**c):
        return
    
    for x in xrange(1000000):
        g(*Nothing())
    
    
    and watch Python's memory use go up and up.
    
    
    Diagnosis:
    
    The analysis begins with the call to PySequence_Tuple at line 1641 in
    ceval.c - the argument to g is seen to be a sequence but not a tuple,
    so it needs to be converted from an abstract sequence to a concrete
    tuple.  PySequence_Tuple starts off by creating a new tuple of length
    5 (line 1122 in abstract.c).  Then at line 1149, since only 3 elements
    were assigned, _PyTuple_Resize is called to make the 5-tuple into a
    3-tuple.  When we're all done the 3-tuple is decrefed, but rather than
    being freed it is placed on the free_tuples cache.
    
    The basic problem is that the 3-tuples are being added to the cache
    but never picked up again, since _PyTuple_Resize doesn't make use of
    the free_tuples cache.  If you are resizing a 5-tuple to a 3-tuple and
    there is already a 3-tuple in free_tuples[3], instead of using this
    tuple, _PyTuple_Resize will realloc the 5-tuple to a 3-tuple.  It
    would more efficient to use the existing 3-tuple and cache the
    5-tuple.
    
    By making _PyTuple_Resize aware of the free_tuples (just as
    PyTuple_New), we not only save a few calls to realloc, but also
    prevent this misbehavior whereby tuples are being added to the
    free_tuples list but never properly "recycled".
    """
    
    And later:
    
    """
    This patch replaces my submission of Sun, 16 Apr and addresses Jeremy
    Hylton's suggestions that we also limit the size of the free tuple
    list.  I chose 2000 as the maximum number of tuples of any particular
    size to save.
    
    There was also a problem with the previous version of this patch
    causing a core dump if Python was built with Py_TRACE_REFS.  This is
    fixed in the below version of the patch, which uses tupledealloc
    instead of _Py_Dealloc.
    """
    b8a9659a
tupleobject.c 12.3 KB