• Marius Gedminas's avatar
    Python 3: pickle bytestrings using SHORT_BINSTRING · a00d35fe
    Marius Gedminas authored
    This uses bytes_as_strings=True option introduced in zodbpickle 0.2 for
    this purpose.
    
    This way pickles produced on Python 3 are nearly the same as on Python 2.
    There are some slight differences (Python 3 seems to perform more
    memoizations which grows the size of some pickles by a couple of bytes),
    but they're immaterial.
    
    Now we can use zodbpickle's noload() on Python 3 to scan pickles for
    persistent references.  We couldn't do that before, because Python 3
    normally pickles byte strings as calls to codecs.encode(u'latin1-data',
    'latin-1'), and noload() doesn't interpret the REDUCE opcode involved in
    that representation.
    
    Note that when you're pickling byte strings using bytes_as_strings=True,
    you have to load them using encoding='bytes' (which breaks instances, so
    cannot be used) or using errors='bytes' (which mean some bytestrings may
    get unpickled as unicode instead).  I've tried hard to discover every
    place that unpickles OIDs and added conversion to bytes in those places.
    
    Applications dealing with binary data be prepared to handle bytestrings
    that unexpectedly become unicode on unpickling.  That's the price of
    Python 2 compatibility.
    a00d35fe
testfsoids.py 7.04 KB