Commit c561a9ad authored by Antoine Pitrou's avatar Antoine Pitrou

Break down and refine memory management question

parent 17bd792c
...@@ -413,66 +413,59 @@ How does Python manage memory? ...@@ -413,66 +413,59 @@ How does Python manage memory?
------------------------------ ------------------------------
The details of Python memory management depend on the implementation. The The details of Python memory management depend on the implementation. The
standard C implementation of Python uses reference counting to detect standard implementation of Python, :term:`CPython`, uses reference counting to
inaccessible objects, and another mechanism to collect reference cycles, detect inaccessible objects, and another mechanism to collect reference cycles,
periodically executing a cycle detection algorithm which looks for inaccessible periodically executing a cycle detection algorithm which looks for inaccessible
cycles and deletes the objects involved. The :mod:`gc` module provides functions cycles and deletes the objects involved. The :mod:`gc` module provides functions
to perform a garbage collection, obtain debugging statistics, and tune the to perform a garbage collection, obtain debugging statistics, and tune the
collector's parameters. collector's parameters.
Jython relies on the Java runtime so the JVM's garbage collector is used. This Other implementations (such as `Jython <http://www.jython.org>`_ or
difference can cause some subtle porting problems if your Python code depends on `PyPy <http://www.pypy.org>`_), however, can rely on a different mechanism
the behavior of the reference counting implementation. such as a full-blown garbage collector. This difference can cause some
subtle porting problems if your Python code depends on the behavior of the
reference counting implementation.
.. XXX relevant for Python 3? In some Python implementations, the following code (which is fine in CPython)
will probably run out of file descriptors::
Sometimes objects get stuck in traceback temporarily and hence are not for file in very_long_list_of_files:
deallocated when you might expect. Clear the traceback with:: f = open(file)
c = f.read(1)
Indeed, using CPython's reference counting and destructor scheme, each new
assignment to *f* closes the previous file. With a traditional GC, however,
those file objects will only get collected (and closed) at varying and possibly
long intervals.
If you want to write code that will work with any Python implementation,
you should explicitly close the file or use the :keyword:`with` statement;
this will work regardless of memory management scheme::
import sys for file in very_long_list_of_files:
sys.last_traceback = None with open(file) as f:
c = f.read(1)
Tracebacks are used for reporting errors, implementing debuggers and related
things. They contain a portion of the program state extracted during the
handling of an exception (usually the most recent exception).
In the absence of circularities, Python programs do not need to manage memory Why doesn't CPython use a more traditional garbage collection scheme?
explicitly. ---------------------------------------------------------------------
Why doesn't Python use a more traditional garbage collection scheme? For one For one thing, this is not a C standard feature and hence it's not portable.
thing, this is not a C standard feature and hence it's not portable. (Yes, we (Yes, we know about the Boehm GC library. It has bits of assembler code for
know about the Boehm GC library. It has bits of assembler code for *most* *most* common platforms, not for all of them, and although it is mostly
common platforms, not for all of them, and although it is mostly transparent, it transparent, it isn't completely transparent; patches are required to get
isn't completely transparent; patches are required to get Python to work with Python to work with it.)
it.)
Traditional GC also becomes a problem when Python is embedded into other Traditional GC also becomes a problem when Python is embedded into other
applications. While in a standalone Python it's fine to replace the standard applications. While in a standalone Python it's fine to replace the standard
malloc() and free() with versions provided by the GC library, an application malloc() and free() with versions provided by the GC library, an application
embedding Python may want to have its *own* substitute for malloc() and free(), embedding Python may want to have its *own* substitute for malloc() and free(),
and may not want Python's. Right now, Python works with anything that and may not want Python's. Right now, CPython works with anything that
implements malloc() and free() properly. implements malloc() and free() properly.
In Jython, the following code (which is fine in CPython) will probably run out
of file descriptors long before it runs out of memory::
for file in very_long_list_of_files:
f = open(file)
c = f.read(1)
Using the current reference counting and destructor scheme, each new assignment
to f closes the previous file. Using GC, this is not guaranteed. If you want
to write code that will work with any Python implementation, you should
explicitly close the file or use the :keyword:`with` statement; this will work
regardless of GC::
for file in very_long_list_of_files:
with open(file) as f:
c = f.read(1)
Why isn't all memory freed when CPython exits?
Why isn't all memory freed when Python exits? ----------------------------------------------
---------------------------------------------
Objects referenced from the global namespaces of Python modules are not always Objects referenced from the global namespaces of Python modules are not always
deallocated when Python exits. This may happen if there are circular deallocated when Python exits. This may happen if there are circular
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment