encoder: Fix string wrt protocol version
- we can use BINSTRING* only if protocol >= 1; - at protocol 0 we thus have to use text STRING; - if protocol >= 3 we have to emit the string as unicode pickle object the same way as Python3 does. If we don't do - Python3 won't be generally able to load our pickle: In [1]: s = b'U\x06\xd0\xbc\xd0\xb8\xd1\x80q\x00.' In [2]: from pickletools import dis In [3]: dis(s) 0: U SHORT_BINSTRING 'миÑ\x80' 8: q BINPUT 0 10: . STOP highest protocol among opcodes = 1 In [4]: import pickle In [5]: pickle.loads(s) --------------------------------------------------------------------------- UnicodeDecodeError Traceback (most recent call last) <ipython-input-5-764e4625bc41> in <module>() ----> 1 pickle.loads(s) UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 0: ordinal not in range(128) We already decode unicode pickle objects into string, this way decode(encode(string)) remains always idempotent.
Showing