• Kirill Smelkov's avatar
    go/zodb/zeo: Fix ZRPC/pickle binary and string encoding/decoding wrt py3 · 50f86b06
    Kirill Smelkov authored
    ZEO/go client was asserting "method" field to be bytestring. As the
    result interoperation with ZEO/py3 server was failing as e.g.
    
        2024-07-18T20:41:14 INFO ZEO.asyncio.server received handshake 'Z5'
        2024/07/18 20:41:14 zlink @ - /tmp/zeo2281880331/1.fs.zeosock: serveRecv1: decode: .0: method: got string; expected str
            zeo_test.go:221: openzeo /tmp/zeo2281880331/1.fs.zeosock: zeo: open /tmp/zeo2281880331/1.fs.zeosock: zlink @ - /tmp/zeo2281880331/1.fs.zeosock: call register: zlink is closed
    
    because ZEO/py3 sends "method" as unicode.
    
    -> Fix it by using pickle.AsString when decoding "method" field.
    
    Similarly ZEO/go client was sending strings as bytestr, which on py3 decode to
    unicode ok only if they are ASCII. Adjust string encoding to be emitted
    as unicode always which works wrt both ZEO/py3 and ZEO/py2.
    
    --------
    
    ZEO/go client was sending binary ZRPC arguments as bytestr as well,
    which was resulting in the following exception on ZEO/py3 side:
    
        2024-07-18T20:42:31 ERROR ZEO.asyncio.server Can't deserialize message
        Traceback (most recent call last):
          File "/home/kirr/src/wendelin/z/ZEO/src/ZEO/asyncio/server.py", line 102, in message_received
            message_id, async_, name, args = self.decode(message)
          File "/home/kirr/src/wendelin/z/ZEO/src/ZEO/asyncio/marshal.py", line 122, in pickle_server_decode
            return unpickler.load()  # msgid, flags, name, args
        UnicodeDecodeError: 'ascii' codec can't decode byte 0x85 in position 1: ordinal not in range(128)
        2024/07/18 20:42:31 zlink @ - /tmp/zeo627526011/1.fs.zeosock: recvPkt: EOF
            xtesting.go:361: load 0285cbac70a3d733:0000000000000001: returned err unexpected:
                have: /tmp/zeo627526011/1.fs.zeosock: load 0285cbac70a3d733:0000000000000001: zlink @ - /tmp/zeo627526011/1.fs.zeosock: call loadBefore: zlink is closed
                want: nil
    
    -> Fix it by using pickle.Bytes which should be encoded as bytes in the
       pickle stream.
    
    This fix is ok but depends on the next patch to upgrade ZODB
    serialization protocol to 3, because on pickle protocol 2 bytes are
    serialized as `_codecs.encode(byt.decode('latin1'), 'latin1')` and
    ZEO/py forbids import of _codecs:
    
        2024-07-18T20:44:39 ERROR ZEO.asyncio.server Can't deserialize message
        Traceback (most recent call last):
          File "/home/kirr/src/wendelin/z/ZEO5/src/ZEO/asyncio/server.py", line 100, in message_received
            message_id, async_, name, args = self.decode(message)
          File "/home/kirr/src/wendelin/z/ZEO5/src/ZEO/asyncio/marshal.py", line 119, in pickle_server_decode
            return unpickler.load()  # msgid, flags, name, args
          File "/home/kirr/src/wendelin/z/ZEO5/src/ZEO/asyncio/marshal.py", line 176, in server_find_global
            raise ImportError("Module not allowed: %s" % (module,))
        ImportError: Module not allowed: _codecs
    
    We will address this in the next patch.
    50f86b06
proto.go 10.6 KB