1. 21 Jul, 2016 1 commit
    • Kirill Smelkov's avatar
      On deactivate release in-slots objects too · fe2219f4
      Kirill Smelkov authored
      On ._p_deactivate() and ._p_invalidate(), when an object goes to ghost
      state, objects referenced by all its attributes, except related to
      persistence machinery, are released, this way freeing memory (if they
      were referenced only from going-to-ghost object).
      
      That's the idea - an object in ghost state is simply a stub, which loads
      its content on first access (via hooking into get/set attr) while
      occupying minimal memory in not-yet-loaded state.
      
      However the above is not completely true right now, as currently on
      ghostification only object's .__dict__ is released, while in-slots objects
      are retained attached to ghost object staying in RAM:
      
          ---- 8< ----
          from ZODB import DB
          from persistent import Persistent
          import gc
      
          db = DB(None)
          jar = db.open()
      
          class C:
              def __init__(self, v):
                  self.v = v
              def __del__(self):
                  print 'released (%s)' % self.v
      
          class P1(Persistent):
              pass
      
          class P2(Persistent):
              __slots__ = ('aaa')
      
          p1 = P1()
          jar.add(p1)
          p1.aaa = C(1)
      
          p2 = P2()
          jar.add(p2)
          p2.aaa = C(2)
      
          p1._p_invalidate()
          # "released (1)" is printed
      
          p2._p_invalidate()
          gc.collect()
          # "released (2)" is NOT printed     <--
          ---- 8< ----
      
      So teach ghostify() & friends to release objects in slots to free-up
      memory when an object goes to ghost state.
      
      NOTE PyErr_Occurred() added after ghostify() calls because
      pickle_slotnames() can raise an error, but we do not want to change
      ghostify() prototype for backward compatibility reason - as it is used
      in cPersistenceCAPIstruct.
      
      ( I hit this bug with wendelin.core which uses proxies to load
        data from DB to virtual memory manager and then deactivate proxy right
        after load has been completed:
      
        https://lab.nexedi.com/nexedi/wendelin.core/blob/f7803634/bigfile/file_zodb.py#L239
        https://lab.nexedi.com/nexedi/wendelin.core/blob/f7803634/bigfile/file_zodb.py#L295 )
      fe2219f4
  2. 26 May, 2016 5 commits
  3. 24 May, 2016 1 commit
  4. 22 May, 2016 1 commit
  5. 18 May, 2016 3 commits
  6. 05 May, 2016 3 commits
  7. 22 Apr, 2016 1 commit
  8. 20 Apr, 2016 1 commit
  9. 15 Apr, 2016 3 commits
  10. 14 Apr, 2016 3 commits
  11. 11 Apr, 2016 1 commit
  12. 25 Mar, 2016 4 commits
  13. 08 Nov, 2015 1 commit
  14. 02 Jun, 2015 3 commits
  15. 19 May, 2015 9 commits