Commit 9ea89d2a authored by Tim Peters's avatar Tim Peters

In a PYMALLOC_DEBUG build obmalloc adds extra debugging info

to each allocated block.  This was using 4 bytes for each such
piece of info regardless of platform.  This didn't really matter
before (proof: no bug reports, and the debug-build obmalloc would
have assert-failed if it was ever asked for a chunk of memory
>= 2**32 bytes), since container indices were plain ints.  But after
the Py_ssize_t changes, it's at least theoretically possible to
allocate a list or string whose guts exceed 2**32 bytes, and the
PYMALLOC_DEBUG routines would fail then (having only 4 bytes
to record the originally requested size).

Now we use sizeof(size_t) bytes for each of a PYMALLOC_DEBUG
build's extra debugging fields.  This won't make any difference
on 32-bit boxes, but will add 16 bytes to each allocation in
a debug build on a 64-bit box.
parent c65a13f5
...@@ -464,7 +464,13 @@ Core and builtins ...@@ -464,7 +464,13 @@ Core and builtins
Note: Codec packages should implement and register their own Note: Codec packages should implement and register their own
codec search function. PEP 100 has the details. codec search function. PEP 100 has the details.
- PEP 353: Using ssize_t as the index type. - PEP 353: Using ``Py_ssize_t`` as the index type.
- ``PYMALLOC_DEBUG`` builds now add ``4*sizeof(size_t)`` bytes of debugging
info to each allocated block, since the ``Py_ssize_t`` changes (PEP 353)
now allow Python to make use of memory blocks exceeding 2**32 bytes for
some purposes on 64-bit boxes. A ``PYMALLOC_DEBUG`` build was limited
to 4-byte allocations before.
- Patch #1400181, fix unicode string formatting to not use the locale. - Patch #1400181, fix unicode string formatting to not use the locale.
This is how string objects work. u'%f' could use , instead of . This is how string objects work. u'%f' could use , instead of .
......
...@@ -96,16 +96,16 @@ dynamically allocated memory blocks. The special bit patterns are: ...@@ -96,16 +96,16 @@ dynamically allocated memory blocks. The special bit patterns are:
Strings of these bytes are unlikely to be valid addresses, floats, or 7-bit Strings of these bytes are unlikely to be valid addresses, floats, or 7-bit
ASCII strings. ASCII strings.
8 bytes are added at each end of each block of N bytes requested. The Let S = sizeof(size_t). 2*S bytes are added at each end of each block of N
memory layout is like so, where p represents the address returned by a bytes requested. The memory layout is like so, where p represents the
malloc-like or realloc-like function (p[i:j] means the slice of bytes address returned by a malloc-like or realloc-like function (p[i:j] means
from *(p+i) inclusive up to *(p+j) exclusive; note that the treatment the slice of bytes from *(p+i) inclusive up to *(p+j) exclusive; note that
of negative indices differs from a Python slice): the treatment of negative indices differs from a Python slice):
p[-8:-4] p[-2*S:-S]
Number of bytes originally asked for. 4-byte unsigned integer, Number of bytes originally asked for. This is a size_t, big-endian
big-endian (easier to read in a memory dump). (easier to read in a memory dump).
p[-4:0] p[-S:0]
Copies of FORBIDDENBYTE. Used to catch under- writes and reads. Copies of FORBIDDENBYTE. Used to catch under- writes and reads.
p[0:N] p[0:N]
The requested memory, filled with copies of CLEANBYTE, used to catch The requested memory, filled with copies of CLEANBYTE, used to catch
...@@ -116,12 +116,12 @@ p[0:N] ...@@ -116,12 +116,12 @@ p[0:N]
DEADBYTE, to catch reference to freed memory. When a realloc- DEADBYTE, to catch reference to freed memory. When a realloc-
like function is called requesting a smaller memory block, the excess like function is called requesting a smaller memory block, the excess
old bytes are also filled with DEADBYTE. old bytes are also filled with DEADBYTE.
p[N:N+4] p[N:N+S]
Copies of FORBIDDENBYTE. Used to catch over- writes and reads. Copies of FORBIDDENBYTE. Used to catch over- writes and reads.
p[N+4:N+8] p[N+S:N+2*S]
A serial number, incremented by 1 on each call to a malloc-like or A serial number, incremented by 1 on each call to a malloc-like or
realloc-like function. realloc-like function.
4-byte unsigned integer, big-endian. Big-endian size_t.
If "bad memory" is detected later, the serial number gives an If "bad memory" is detected later, the serial number gives an
excellent way to set a breakpoint on the next run, to capture the excellent way to set a breakpoint on the next run, to capture the
instant at which this block was passed out. The static function instant at which this block was passed out. The static function
...@@ -145,6 +145,10 @@ envar PYTHONMALLOCSTATS ...@@ -145,6 +145,10 @@ envar PYTHONMALLOCSTATS
If this envar exists, a report of pymalloc summary statistics is If this envar exists, a report of pymalloc summary statistics is
printed to stderr whenever a new arena is allocated, and also printed to stderr whenever a new arena is allocated, and also
by Py_Finalize(). by Py_Finalize().
Changed in 2.5: The number of extra bytes allocated is 4*sizeof(size_t).
Before it was 16 on all boxes, reflecting that Python couldn't make use of
allocations >= 2**32 bytes even on 64-bit boxes before 2.5.
--------------------------------------------------------------------------- ---------------------------------------------------------------------------
Py_DEBUG introduced in 1.5 Py_DEBUG introduced in 1.5
named DEBUG before 1.5 named DEBUG before 1.5
...@@ -251,7 +255,7 @@ is not defined by the architecture specification, so you'll need to ...@@ -251,7 +255,7 @@ is not defined by the architecture specification, so you'll need to
find the manual for your specific processor. For the 750CX, 750CXe find the manual for your specific processor. For the 750CX, 750CXe
and 750FX (all sold as the G3) we find: and 750FX (all sold as the G3) we find:
The time base counter is clocked at a frequency that is The time base counter is clocked at a frequency that is
one-fourth that of the bus clock. one-fourth that of the bus clock.
This build is enabled by the --with-tsc flag to configure. This build is enabled by the --with-tsc flag to configure.
This diff is collapsed.
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment