Commit 29bf27fe authored by Victor Stinner's avatar Victor Stinner

Issue #26516: Enhance Python mem allocators doc

* add link to PYTHONMALLOCSTATS env var
* add parameters to PyMem macros like PyMem_MALLOC()
* fix PyMem_SetupDebugHooks(): add Calloc functions
* add some newlines for readability
parent 42a4366a
...@@ -83,6 +83,12 @@ collection, memory compaction or other preventive procedures. Note that by using ...@@ -83,6 +83,12 @@ collection, memory compaction or other preventive procedures. Note that by using
the C library allocator as shown in the previous example, the allocated memory the C library allocator as shown in the previous example, the allocated memory
for the I/O buffer escapes completely the Python memory manager. for the I/O buffer escapes completely the Python memory manager.
.. seealso::
The :envvar:`PYTHONMALLOCSTATS` environment variable can be used to print
memory allocation statistics every time a new object arena is created, and
on shutdown.
Raw Memory Interface Raw Memory Interface
==================== ====================
...@@ -100,9 +106,10 @@ The default raw memory block allocator uses the following functions: ...@@ -100,9 +106,10 @@ The default raw memory block allocator uses the following functions:
.. c:function:: void* PyMem_RawMalloc(size_t n) .. c:function:: void* PyMem_RawMalloc(size_t n)
Allocates *n* bytes and returns a pointer of type :c:type:`void\*` to the Allocates *n* bytes and returns a pointer of type :c:type:`void\*` to the
allocated memory, or *NULL* if the request fails. Requesting zero bytes allocated memory, or *NULL* if the request fails.
returns a distinct non-*NULL* pointer if possible, as if
``PyMem_RawMalloc(1)`` had been called instead. The memory will not have Requesting zero bytes returns a distinct non-*NULL* pointer if possible, as
if ``PyMem_RawMalloc(1)`` had been called instead. The memory will not have
been initialized in any way. been initialized in any way.
...@@ -110,9 +117,11 @@ The default raw memory block allocator uses the following functions: ...@@ -110,9 +117,11 @@ The default raw memory block allocator uses the following functions:
Allocates *nelem* elements each whose size in bytes is *elsize* and returns Allocates *nelem* elements each whose size in bytes is *elsize* and returns
a pointer of type :c:type:`void\*` to the allocated memory, or *NULL* if the a pointer of type :c:type:`void\*` to the allocated memory, or *NULL* if the
request fails. The memory is initialized to zeros. Requesting zero elements request fails. The memory is initialized to zeros.
or elements of size zero bytes returns a distinct non-*NULL* pointer if
possible, as if ``PyMem_RawCalloc(1, 1)`` had been called instead. Requesting zero elements or elements of size zero bytes returns a distinct
non-*NULL* pointer if possible, as if ``PyMem_RawCalloc(1, 1)`` had been
called instead.
.. versionadded:: 3.5 .. versionadded:: 3.5
...@@ -120,21 +129,28 @@ The default raw memory block allocator uses the following functions: ...@@ -120,21 +129,28 @@ The default raw memory block allocator uses the following functions:
.. c:function:: void* PyMem_RawRealloc(void *p, size_t n) .. c:function:: void* PyMem_RawRealloc(void *p, size_t n)
Resizes the memory block pointed to by *p* to *n* bytes. The contents will Resizes the memory block pointed to by *p* to *n* bytes. The contents will
be unchanged to the minimum of the old and the new sizes. If *p* is *NULL*, be unchanged to the minimum of the old and the new sizes.
the call is equivalent to ``PyMem_RawMalloc(n)``; else if *n* is equal to
zero, the memory block is resized but is not freed, and the returned pointer If *p* is *NULL*, the call is equivalent to ``PyMem_RawMalloc(n)``; else if
is non-*NULL*. Unless *p* is *NULL*, it must have been returned by a *n* is equal to zero, the memory block is resized but is not freed, and the
previous call to :c:func:`PyMem_RawMalloc` or :c:func:`PyMem_RawRealloc`. If returned pointer is non-*NULL*.
the request fails, :c:func:`PyMem_RawRealloc` returns *NULL* and *p* remains
a valid pointer to the previous memory area. Unless *p* is *NULL*, it must have been returned by a previous call to
:c:func:`PyMem_RawMalloc`, :c:func:`PyMem_RawRealloc` or
:c:func:`PyMem_RawCalloc`.
If the request fails, :c:func:`PyMem_RawRealloc` returns *NULL* and *p*
remains a valid pointer to the previous memory area.
.. c:function:: void PyMem_RawFree(void *p) .. c:function:: void PyMem_RawFree(void *p)
Frees the memory block pointed to by *p*, which must have been returned by a Frees the memory block pointed to by *p*, which must have been returned by a
previous call to :c:func:`PyMem_RawMalloc` or :c:func:`PyMem_RawRealloc`. previous call to :c:func:`PyMem_RawMalloc`, :c:func:`PyMem_RawRealloc` or
Otherwise, or if ``PyMem_Free(p)`` has been called before, undefined :c:func:`PyMem_RawCalloc`. Otherwise, or if ``PyMem_Free(p)`` has been
behavior occurs. If *p* is *NULL*, no operation is performed. called before, undefined behavior occurs.
If *p* is *NULL*, no operation is performed.
.. _memoryinterface: .. _memoryinterface:
...@@ -158,18 +174,22 @@ The default memory block allocator uses the following functions: ...@@ -158,18 +174,22 @@ The default memory block allocator uses the following functions:
.. c:function:: void* PyMem_Malloc(size_t n) .. c:function:: void* PyMem_Malloc(size_t n)
Allocates *n* bytes and returns a pointer of type :c:type:`void\*` to the Allocates *n* bytes and returns a pointer of type :c:type:`void\*` to the
allocated memory, or *NULL* if the request fails. Requesting zero bytes returns allocated memory, or *NULL* if the request fails.
a distinct non-*NULL* pointer if possible, as if ``PyMem_Malloc(1)`` had
been called instead. The memory will not have been initialized in any way. Requesting zero bytes returns a distinct non-*NULL* pointer if possible, as
if ``PyMem_Malloc(1)`` had been called instead. The memory will not have
been initialized in any way.
.. c:function:: void* PyMem_Calloc(size_t nelem, size_t elsize) .. c:function:: void* PyMem_Calloc(size_t nelem, size_t elsize)
Allocates *nelem* elements each whose size in bytes is *elsize* and returns Allocates *nelem* elements each whose size in bytes is *elsize* and returns
a pointer of type :c:type:`void\*` to the allocated memory, or *NULL* if the a pointer of type :c:type:`void\*` to the allocated memory, or *NULL* if the
request fails. The memory is initialized to zeros. Requesting zero elements request fails. The memory is initialized to zeros.
or elements of size zero bytes returns a distinct non-*NULL* pointer if
possible, as if ``PyMem_Calloc(1, 1)`` had been called instead. Requesting zero elements or elements of size zero bytes returns a distinct
non-*NULL* pointer if possible, as if ``PyMem_Calloc(1, 1)`` had been called
instead.
.. versionadded:: 3.5 .. versionadded:: 3.5
...@@ -177,21 +197,27 @@ The default memory block allocator uses the following functions: ...@@ -177,21 +197,27 @@ The default memory block allocator uses the following functions:
.. c:function:: void* PyMem_Realloc(void *p, size_t n) .. c:function:: void* PyMem_Realloc(void *p, size_t n)
Resizes the memory block pointed to by *p* to *n* bytes. The contents will be Resizes the memory block pointed to by *p* to *n* bytes. The contents will be
unchanged to the minimum of the old and the new sizes. If *p* is *NULL*, the unchanged to the minimum of the old and the new sizes.
call is equivalent to ``PyMem_Malloc(n)``; else if *n* is equal to zero,
the memory block is resized but is not freed, and the returned pointer is If *p* is *NULL*, the call is equivalent to ``PyMem_Malloc(n)``; else if *n*
non-*NULL*. Unless *p* is *NULL*, it must have been returned by a previous call is equal to zero, the memory block is resized but is not freed, and the
to :c:func:`PyMem_Malloc` or :c:func:`PyMem_Realloc`. If the request fails, returned pointer is non-*NULL*.
:c:func:`PyMem_Realloc` returns *NULL* and *p* remains a valid pointer to the
previous memory area. Unless *p* is *NULL*, it must have been returned by a previous call to
:c:func:`PyMem_Malloc`, :c:func:`PyMem_Realloc` or :c:func:`PyMem_Calloc`.
If the request fails, :c:func:`PyMem_Realloc` returns *NULL* and *p* remains
a valid pointer to the previous memory area.
.. c:function:: void PyMem_Free(void *p) .. c:function:: void PyMem_Free(void *p)
Frees the memory block pointed to by *p*, which must have been returned by a Frees the memory block pointed to by *p*, which must have been returned by a
previous call to :c:func:`PyMem_Malloc` or :c:func:`PyMem_Realloc`. Otherwise, or previous call to :c:func:`PyMem_Malloc`, :c:func:`PyMem_Realloc` or
if ``PyMem_Free(p)`` has been called before, undefined behavior occurs. If :c:func:`PyMem_Calloc`. Otherwise, or if ``PyMem_Free(p)`` has been called
*p* is *NULL*, no operation is performed. before, undefined behavior occurs.
If *p* is *NULL*, no operation is performed.
The following type-oriented macros are provided for convenience. Note that The following type-oriented macros are provided for convenience. Note that
*TYPE* refers to any C type. *TYPE* refers to any C type.
...@@ -209,8 +235,10 @@ The following type-oriented macros are provided for convenience. Note that ...@@ -209,8 +235,10 @@ The following type-oriented macros are provided for convenience. Note that
Same as :c:func:`PyMem_Realloc`, but the memory block is resized to ``(n * Same as :c:func:`PyMem_Realloc`, but the memory block is resized to ``(n *
sizeof(TYPE))`` bytes. Returns a pointer cast to :c:type:`TYPE\*`. On return, sizeof(TYPE))`` bytes. Returns a pointer cast to :c:type:`TYPE\*`. On return,
*p* will be a pointer to the new memory area, or *NULL* in the event of *p* will be a pointer to the new memory area, or *NULL* in the event of
failure. This is a C preprocessor macro; p is always reassigned. Save failure.
the original value of p to avoid losing memory when handling errors.
This is a C preprocessor macro; *p* is always reassigned. Save the original
value of *p* to avoid losing memory when handling errors.
.. c:function:: void PyMem_Del(void *p) .. c:function:: void PyMem_Del(void *p)
...@@ -222,9 +250,12 @@ allocator directly, without involving the C API functions listed above. However, ...@@ -222,9 +250,12 @@ allocator directly, without involving the C API functions listed above. However,
note that their use does not preserve binary compatibility across Python note that their use does not preserve binary compatibility across Python
versions and is therefore deprecated in extension modules. versions and is therefore deprecated in extension modules.
:c:func:`PyMem_MALLOC`, :c:func:`PyMem_REALLOC`, :c:func:`PyMem_FREE`. * ``PyMem_MALLOC(size)``
* ``PyMem_NEW(type, size)``
:c:func:`PyMem_NEW`, :c:func:`PyMem_RESIZE`, :c:func:`PyMem_DEL`. * ``PyMem_REALLOC(ptr, size)``
* ``PyMem_RESIZE(ptr, type, size)``
* ``PyMem_FREE(ptr)``
* ``PyMem_DEL(ptr)``
Customize Memory Allocators Customize Memory Allocators
...@@ -262,11 +293,13 @@ Customize Memory Allocators ...@@ -262,11 +293,13 @@ Customize Memory Allocators
Enum used to identify an allocator domain. Domains: Enum used to identify an allocator domain. Domains:
* :c:data:`PYMEM_DOMAIN_RAW`: functions :c:func:`PyMem_RawMalloc`, * :c:data:`PYMEM_DOMAIN_RAW`: functions :c:func:`PyMem_RawMalloc`,
:c:func:`PyMem_RawRealloc` and :c:func:`PyMem_RawFree` :c:func:`PyMem_RawRealloc`, :c:func:`PyMem_RawCalloc` and
:c:func:`PyMem_RawFree`
* :c:data:`PYMEM_DOMAIN_MEM`: functions :c:func:`PyMem_Malloc`, * :c:data:`PYMEM_DOMAIN_MEM`: functions :c:func:`PyMem_Malloc`,
:c:func:`PyMem_Realloc` and :c:func:`PyMem_Free` :c:func:`PyMem_Realloc`, :c:func:`PyMem_Calloc` and :c:func:`PyMem_Free`
* :c:data:`PYMEM_DOMAIN_OBJ`: functions :c:func:`PyObject_Malloc`, * :c:data:`PYMEM_DOMAIN_OBJ`: functions :c:func:`PyObject_Malloc`,
:c:func:`PyObject_Realloc` and :c:func:`PyObject_Free` :c:func:`PyObject_Realloc`, :c:func:`PyObject_Calloc` and
:c:func:`PyObject_Free`
.. c:function:: void PyMem_GetAllocator(PyMemAllocatorDomain domain, PyMemAllocatorEx *allocator) .. c:function:: void PyMem_GetAllocator(PyMemAllocatorDomain domain, PyMemAllocatorEx *allocator)
...@@ -296,10 +329,11 @@ Customize Memory Allocators ...@@ -296,10 +329,11 @@ Customize Memory Allocators
functions: functions:
- :c:func:`PyMem_RawMalloc`, :c:func:`PyMem_RawRealloc`, - :c:func:`PyMem_RawMalloc`, :c:func:`PyMem_RawRealloc`,
:c:func:`PyMem_RawFree` :c:func:`PyMem_RawCalloc`, :c:func:`PyMem_RawFree`
- :c:func:`PyMem_Malloc`, :c:func:`PyMem_Realloc`, :c:func:`PyMem_Free` - :c:func:`PyMem_Malloc`, :c:func:`PyMem_Realloc`, :c:func:`PyMem_Calloc`,
:c:func:`PyMem_Free`
- :c:func:`PyObject_Malloc`, :c:func:`PyObject_Realloc`, - :c:func:`PyObject_Malloc`, :c:func:`PyObject_Realloc`,
:c:func:`PyObject_Free` :c:func:`PyObject_Calloc`, :c:func:`PyObject_Free`
Newly allocated memory is filled with the byte ``0xCB``, freed memory is Newly allocated memory is filled with the byte ``0xCB``, freed memory is
filled with the byte ``0xDB``. Additional checks: filled with the byte ``0xDB``. Additional checks:
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment