With upcoming msgpack encoding packets might be with data < PktHeaderSize.
Adjust pktAlloc to unconditionally provide the invariant.

Preparatory step before we introduce support for msgpack encoding.

It is not used anywhere and would have to be reworked with upcoming
introduction of msgpack encoding. -> Simply remove that.

Just renaming.

This represents a neonet testing environment and in the future will be
used to run tests under different protocol versions, protocol encodings, etc.

For now it is noop wrapper around testing.T + t.bin that wraps []byte
and will be amended to return something different depending on t's
encoding.

Previously we were doing handshake symmetrically: both client and server
were transmitting hello and receiving peer's hello simultaneously.
However this does not allow server to adjust its behaviour depending on
which client (protocol version, protocol encoding, ...) is connecting to it.

-> Rework handshake so that client always sends its hello first, and
only then the server side replies. This matches actual NEO/py behaviour:

https://lab.nexedi.com/nexedi/neoppod/blob/v1.12-67-g261dd4b4/neo/lib/connector.py#L293-294

even though the "NEO protocol" states that

	Handshake transmissions are not ordered with respect to each other and can go in parallel.

	( https://neo.nexedi.com/P-NEO-Protocol.Specification.2019?portal_skin=CI_slideshow#/9/2 )

If I recall correctly that sentence was authored by me in 2018 based on
previous understanding of should-be full symmetry in-between client and
server.

However soon we are going to teach server sides to autodetect client
encoding and adjust server to talk to client via its preferred way.
This needs handshake for client and server to be differentiated.

The protocol needs to be adjusted as well. However I'm not sure it is
going to happen...

This will be handy to test underlying error for e.g. EOF with
errors.Cause or errors.Is in case that underlying error is also error
wrapper.

We were returning packet with tail read for next packet data.
Previously decoding was forgiving, but upcoming rework for msgpack
support will add check to catch packets with overlong payload, e.g.

    === RUN   TestEmptyDB/py/!ssl
    I: runneo.py: /tmp/neo214694750/1 !ssl: started master(s): 127.0.0.1:30621
    127.0.0.1:42266 > 127.0.0.1:30621: .1 RequestIdentification &{CLIENT ?(0)0  1 ø [] []}
    127.0.0.1:42266 < 127.0.0.1:30621: (N: decode header: len(payload) != msgLen) 00 00 00 01 80 01 00 00 00 09 00 f0 00 00 01 e0 00 00 01 00 00 00 00 00 06 00 00 00 58 41 d7 ff 69 82 0a 91 25 00 00 00 03 02 00 00 00 00 e0 00 00 01 02 41 d7 ff 69 82 0a 78 ff 01 00 00 00 09 31 32 37 2e 30 2e 30 2e 31 96 f7 00 00 00 01 02 41 d7 ff 69 81 c6 65 f5 00 00 00 00 09 31 32 37 2e 30 2e 30 2e 31 77 9d f0 00 00 01 02 ff ff ff ff ff ff ff ff 00 00 00 02 00 0a 00 00 00 19 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 01 00 00 00 01 01

-> Fix is: pkt.data should be data[:pktLen], not data[:n] since n is how
much we have read at all.

The utility to create Networker suitable for interoperating with nodes
in a NEO cluster, be it TCP, TLS/TCP, TLS/lonet, etc. See added comments
for details.

See ConfigForP2P comment for details.

xnet.Networker added Close recently and is now required to be explicitly
closed by the user:

kirr/go123@01a77bb9

See kirr/go123@515a6d14

See

	kirr/go123@3354b401
and

	kirr/go123@b03d65ff

The wrapping logic in LinkListener goes away because Accept from xnet
now supports cancellation.

It was not the case with ZEO4 server because we did not had
kirr/ZEO@bf80d23d yet.

For ZEO it is not strictly required, but for upcoming NEO for example
NEOCluster needs to shutdown gracefully, else there are processes left for e.g.
storage nodes and they dump somthing as below on the terminal after tests
completion:

    === RUN   TestLoad
    2020/10/21 14:33:00 zodb: FIXME: open ../zodb/storage/fs1/testdata/1.fs: raw cache is not ready for invalidations -> NoCache forced
    === RUN   TestLoad/py
    I: runneo.py: /tmp/neo445013868/1: Started master(s): 127.0.0.1:24661
    WARNING: This is not the recommended way to import data to NEO: you should use the Importer backend instead.
    NEO also does not implement IStorageRestoreable interface, which means that undo information is not preserved when using this tool: conflict resolution could happen when undoing an old transaction.
    Migrating from ../zodb/storage/fs1/testdata/1.fs to 127.0.0.1:24661
    Migration done in 0.19877
    --- PASS: TestLoad (0.75s)
        --- PASS: TestLoad/py (0.74s)
    PASS
    ok      lab.nexedi.com/kirr/neo/go/neo  0.749s
    (neo) (z-dev) (g.env) kirr@deco:~/src/neo/src/lab.nexedi.com/kirr/neo/go/neo$ Traceback (most recent call last):
      File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/tests/functional/__init__.py", line 182, in start
        getattr(neo.scripts,  command).main()
      File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/scripts/neostorage.py", line 66, in main
        app.run()
      File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/storage/app.py", line 147, in run
        self._run()
      File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/storage/app.py", line 178, in _run
        self.doOperation()
      File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/storage/app.py", line 266, in doOperation
        poll()
      File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/storage/app.py", line 87, in _poll
        self.em.poll(1)
      File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/lib/event.py", line 155, in poll
        self._poll(blocking)
      File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/lib/event.py", line 253, in _poll
        timeout_object.onTimeout()
      File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/lib/event.py", line 259, in onTimeout
        on_timeout()
      File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/storage/database/manager.py", line 207, in _deferredCommit
        self.commit()
      File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/storage/database/manager.py", line 193, in commit
        self._commit()
      File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/storage/database/sqlite.py", line 90, in _commit
        retry_if_locked(self.conn.commit)
      File "/home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/neo/storage/database/sqlite.py", line 45, in retry_if_locked
        return f(*args)
    OperationalError: disk I/O error

And use it to verify other storages.

See nexedi/nxdtest@53064e71

Go through zrpc and add error context to every function that can create
error. This is similar to what NEO does in neonet. The reason to do this
was the following obscure error from wcfs:

    E1105 11:32:33.295497   24639 wcfs.go:2505] zwatch zeo://:28359: EOF

which, after this patch, is now

    E1105 12:53:15.922052   30731 wcfs.go:2510] zwatch zeo://:27024: zlink 127.0.0.1:47768 - 127.0.0.1:27024: recvPkt: EOF

NEO was already doing this, but let's use xio.NoEOF everywhere for
uniformity.

If remote peer closes the link half-way of packet data - turn EOF into
ErrUnexpectedEOF.

We will need to use those utilities for ZEO and NEO.

We recently hit "decode: buffer overflow" errors due to mismatch in
between NEO/go and NEO/py (see previous patch). With dumpio=true that
problem was showing itself as

    127.0.0.1:50624 > 127.0.0.1:41863: .5 GetObject &{0000000000000000 ffffffffffffffff 0285cbac258bf266}
    127.0.0.1:50624 < 127.0.0.1:41863: .5 (proto.Error) decode: buffer overflow; #24 [24]: 00 00 00 03 00 00 00 10 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 Error ACK ;  [24]tail: 00 00 00 03 00 00 00 10 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30
        xtesting.go:306: load 0285cbac258bf265:0000000000000000: returned err unexpected:
            have: neo://1@127.0.0.1:24078: load 0285cbac258bf265:0000000000000000: 127.0.0.1:50624 - 127.0.0.1:41863 .5: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:24078: load 0285cbac258bf265:0000000000000000: 0000000000000000: object was not yet created

Here after printing error and dumping all bytes from packet payload, it
was printing again message type, message value from to-be-decoded place
(which is zero-initialized) and data bytes again.
-> Fix it not to print anything after dump of payload data:

    127.0.0.1:60518 > 127.0.0.1:46719: .5 GetObject &{0000000000000000 ffffffffffffffff 0285cbac258bf266}
    127.0.0.1:60518 < 127.0.0.1:46719: .5 (Error) decode: buffer overflow; #24 [24]: 00 00 00 03 00 00 00 10 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30
        xtesting.go:306: load 0285cbac258bf265:0000000000000000: returned err unexpected:
            have: neo://1@127.0.0.1:27853: load 0285cbac258bf265:0000000000000000: 127.0.0.1:60518 - 127.0.0.1:46719 .5: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:27853: load 0285cbac258bf265:0000000000000000: 0000000000000000: object was not yet created

In commit 5f13cc85 I changed enum encodings from int32 to int8, but did
not noticed that NEO/py commit 52db5607 ("protocol: a single byte is
more than enough to encode enums") despite specified intent and
ErrorCodes being marked with @Enum, changed encoding only for fields
that are marked as PEnum in structures. In NEO/py the Error.code field
is still marked as PNumber which encodes as 32-bit integer on the wire.
-> Fix it back.

With recent xtesting.DrvTestLoad update this error was manifesting itself as (on @t branch):

--- FAIL: TestLoad (2.08s)
    --- FAIL: TestLoad/py (2.07s)
        xtesting.go:306: load 0285cbac258bf265:0000000000000000: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbac258bf265:0000000000000000: 127.0.0.1:42288 - 127.0.0.1:37109 .5: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbac258bf265:0000000000000000: 0000000000000000: object was not yet created
        xtesting.go:306: load 0285cbac3d0369e5:0000000000000001: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbac3d0369e5:0000000000000001: 127.0.0.1:42288 - 127.0.0.1:37109 .13: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbac3d0369e5:0000000000000001: 0000000000000001: object was not yet created
        xtesting.go:306: load 0285cbac41b4e832:0000000000000002: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbac41b4e832:0000000000000002: 127.0.0.1:42288 - 127.0.0.1:37109 .21: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbac41b4e832:0000000000000002: 0000000000000002: object was not yet created
        xtesting.go:306: load 0285cbac4666667f:0000000000000003: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbac4666667f:0000000000000003: 127.0.0.1:42288 - 127.0.0.1:37109 .29: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbac4666667f:0000000000000003: 0000000000000003: object was not yet created
        xtesting.go:306: load 0285cbac4fc96318:0000000000000004: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbac4fc96318:0000000000000004: 127.0.0.1:42288 - 127.0.0.1:37109 .41: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbac4fc96318:0000000000000004: 0000000000000004: object was not yet created
        xtesting.go:306: load 0285cbac547ae165:0000000000000005: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbac547ae165:0000000000000005: 127.0.0.1:42288 - 127.0.0.1:37109 .49: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbac547ae165:0000000000000005: 0000000000000005: object was not yet created
        xtesting.go:306: load 0285cbac628f5c4b:0000000000000006: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbac628f5c4b:0000000000000006: 127.0.0.1:42288 - 127.0.0.1:37109 .65: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbac628f5c4b:0000000000000006: 0000000000000006: object was not yet created
        xtesting.go:306: load 0285cbaca444447f:0000000000000007: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbaca444447f:0000000000000007: 127.0.0.1:42288 - 127.0.0.1:37109 .125: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbaca444447f:0000000000000007: 0000000000000007: object was not yet created
        xtesting.go:306: load 0285cbacbbbbbbff:0000000000000008: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbacbbbbbbff:0000000000000008: 127.0.0.1:42288 - 127.0.0.1:37109 .149: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbacbbbbbbff:0000000000000008: 0000000000000008: object was not yet created
        xtesting.go:306: load 0285cbad80da7498:0000000000000009: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbad80da7498:0000000000000009: 127.0.0.1:42288 - 127.0.0.1:37109 .269: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbad80da7498:0000000000000009: 0000000000000009: object was not yet created
        xtesting.go:331: load 7fffffffffffffff:000000000000000a: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 7fffffffffffffff:000000000000000a: 127.0.0.1:42288 - 127.0.0.1:37109 .295: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 7fffffffffffffff:000000000000000a: 000000000000000a: no such object

Before this patch and with updated DrvTestLoad (see previous patch) it was failing as:

--- FAIL: TestLoad (0.52s)
    --- FAIL: TestLoad/py/msgpack=false (0.25s)
        xtesting.go:306: load 0285cbac258bf265:0000000000000000: returned err unexpected:
            have: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac258bf265:0000000000000000: /tmp/zeo535364855/1.fs.zeosock: call loadBefore: unexpected reply: got ogórek.None{}; expect 3-tuple
            want: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac258bf265:0000000000000000: 0000000000000000: no such object
        xtesting.go:306: load 0285cbac3d0369e5:0000000000000001: returned err unexpected:
            have: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac3d0369e5:0000000000000001: /tmp/zeo535364855/1.fs.zeosock: call loadBefore: unexpected reply: got ogórek.None{}; expect 3-tuple
            want: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac3d0369e5:0000000000000001: 0000000000000001: no such object
        xtesting.go:306: load 0285cbac41b4e832:0000000000000002: returned err unexpected:
            have: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac41b4e832:0000000000000002: /tmp/zeo535364855/1.fs.zeosock: call loadBefore: unexpected reply: got ogórek.None{}; expect 3-tuple
            want: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac41b4e832:0000000000000002: 0000000000000002: no such object
        xtesting.go:306: load 0285cbac4666667f:0000000000000003: returned err unexpected:
            have: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac4666667f:0000000000000003: /tmp/zeo535364855/1.fs.zeosock: call loadBefore: unexpected reply: got ogórek.None{}; expect 3-tuple
            want: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac4666667f:0000000000000003: 0000000000000003: no such object
        xtesting.go:306: load 0285cbac4fc96318:0000000000000004: returned err unexpected:
            have: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac4fc96318:0000000000000004: /tmp/zeo535364855/1.fs.zeosock: call loadBefore: unexpected reply: got ogórek.None{}; expect 3-tuple
            want: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac4fc96318:0000000000000004: 0000000000000004: no such object
        xtesting.go:306: load 0285cbac547ae165:0000000000000005: returned err unexpected:
            have: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac547ae165:0000000000000005: /tmp/zeo535364855/1.fs.zeosock: call loadBefore: unexpected reply: got ogórek.None{}; expect 3-tuple
            want: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac547ae165:0000000000000005: 0000000000000005: no such object
        xtesting.go:306: load 0285cbac628f5c4b:0000000000000006: returned err unexpected:
            have: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac628f5c4b:0000000000000006: /tmp/zeo535364855/1.fs.zeosock: call loadBefore: unexpected reply: got ogórek.None{}; expect 3-tuple
            want: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac628f5c4b:0000000000000006: 0000000000000006: no such object
        xtesting.go:306: load 0285cbaca444447f:0000000000000007: returned err unexpected:
            have: /tmp/zeo535364855/1.fs.zeosock: load 0285cbaca444447f:0000000000000007: /tmp/zeo535364855/1.fs.zeosock: call loadBefore: unexpected reply: got ogórek.None{}; expect 3-tuple
            want: /tmp/zeo535364855/1.fs.zeosock: load 0285cbaca444447f:0000000000000007: 0000000000000007: no such object
        xtesting.go:306: load 0285cbacbbbbbbff:0000000000000008: returned err unexpected:
            have: /tmp/zeo535364855/1.fs.zeosock: load 0285cbacbbbbbbff:0000000000000008: /tmp/zeo535364855/1.fs.zeosock: call loadBefore: unexpected reply: got ogórek.None{}; expect 3-tuple
            want: /tmp/zeo535364855/1.fs.zeosock: load 0285cbacbbbbbbff:0000000000000008: 0000000000000008: no such object
        xtesting.go:306: load 0285cbad80da7498:0000000000000009: returned err unexpected:
            have: /tmp/zeo535364855/1.fs.zeosock: load 0285cbad80da7498:0000000000000009: /tmp/zeo535364855/1.fs.zeosock: call loadBefore: unexpected reply: got ogórek.None{}; expect 3-tuple
            want: /tmp/zeo535364855/1.fs.zeosock: load 0285cbad80da7498:0000000000000009: 0000000000000009: no such object
    --- FAIL: TestLoad/py/msgpack=true (0.26s)
        xtesting.go:306: load 0285cbac258bf265:0000000000000000: returned err unexpected:
            have: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac258bf265:0000000000000000: /tmp/zeo794664426/1.fs.zeosock: call loadBefore: zlink is closed
            want: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac258bf265:0000000000000000: 0000000000000000: no such object
        xtesting.go:306: load 0285cbac3d0369e5:0000000000000001: returned err unexpected:
            have: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac3d0369e5:0000000000000001: /tmp/zeo794664426/1.fs.zeosock: call loadBefore: zlink is closed
            want: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac3d0369e5:0000000000000001: 0000000000000001: no such object
        xtesting.go:306: load 0285cbac41b4e832:0000000000000002: returned err unexpected:
            have: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac41b4e832:0000000000000002: /tmp/zeo794664426/1.fs.zeosock: call loadBefore: zlink is closed
            want: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac41b4e832:0000000000000002: 0000000000000002: no such object
        xtesting.go:306: load 0285cbac4666667f:0000000000000003: returned err unexpected:
            have: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac4666667f:0000000000000003: /tmp/zeo794664426/1.fs.zeosock: call loadBefore: zlink is closed
            want: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac4666667f:0000000000000003: 0000000000000003: no such object
        xtesting.go:306: load 0285cbac4fc96318:0000000000000004: returned err unexpected:
            have: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac4fc96318:0000000000000004: /tmp/zeo794664426/1.fs.zeosock: call loadBefore: zlink is closed
            want: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac4fc96318:0000000000000004: 0000000000000004: no such object
        xtesting.go:306: load 0285cbac547ae165:0000000000000005: returned err unexpected:
            have: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac547ae165:0000000000000005: /tmp/zeo794664426/1.fs.zeosock: call loadBefore: zlink is closed
            want: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac547ae165:0000000000000005: 0000000000000005: no such object
        xtesting.go:306: load 0285cbac628f5c4b:0000000000000006: returned err unexpected:
            have: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac628f5c4b:0000000000000006: /tmp/zeo794664426/1.fs.zeosock: call loadBefore: zlink is closed
            want: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac628f5c4b:0000000000000006: 0000000000000006: no such object
        xtesting.go:306: load 0285cbaca444447f:0000000000000007: returned err unexpected:
            have: /tmp/zeo794664426/1.fs.zeosock: load 0285cbaca444447f:0000000000000007: /tmp/zeo794664426/1.fs.zeosock: call loadBefore: zlink is closed
            want: /tmp/zeo794664426/1.fs.zeosock: load 0285cbaca444447f:0000000000000007: 0000000000000007: no such object
        xtesting.go:306: load 0285cbacbbbbbbff:0000000000000008: returned err unexpected:
            have: /tmp/zeo794664426/1.fs.zeosock: load 0285cbacbbbbbbff:0000000000000008: /tmp/zeo794664426/1.fs.zeosock: call loadBefore: zlink is closed
            want: /tmp/zeo794664426/1.fs.zeosock: load 0285cbacbbbbbbff:0000000000000008: 0000000000000008: no such object
        xtesting.go:306: load 0285cbad80da7498:0000000000000009: returned err unexpected:
            have: /tmp/zeo794664426/1.fs.zeosock: load 0285cbad80da7498:0000000000000009: /tmp/zeo794664426/1.fs.zeosock: call loadBefore: zlink is closed
            want: /tmp/zeo794664426/1.fs.zeosock: load 0285cbad80da7498:0000000000000009: 0000000000000009: no such object

This currently fails on ZEO and NEO. Those storages will be fixed in the
following patches (NEO only on @t branch for now).

Starting from Go1.10 `go test` caches successful test results[1].
However we want to torture the implementation and run the tests for real
each time `neotest test-go` is ran.

-> Disable tests caching

[1] https://golang.org/doc/go1.10#test

See:

- https://lab.nexedi.com/nexedi/nxdtest
- https://lab.nexedi.com/nexedi/nxdtest/blob/master/nxdtest/__init__.py
- nexedi/nxdtest@d575236a
- nexedi/slapos!839

-> Leave only .nxdtest file with neotest-specific bits to be run by
nxdtest driver.

* origin/old-proto:
  qa: skip broken ZODB test
  client: fix race with invalidations when starting a new transaction on ZODB 5
  Code clean-up, comment fixes
  master: fix crash in STARTING_BACKUP when connecting to an upstream secondary master
  mysql: workaround for MDEV-20693
  client: inline Application._loadFromCache
  client: replace global load lock by a per-oid one
  client: unindent code
  client: remove load lock in tpc_finish
  qa: check cache in testExternalInvalidation
  qa: comment testExternalInvalidation2

This protocol version corresponds to protocol version used by NEO/py
v1.12 and was set in NEO/py commit c6453626 (Bump protocol version).

The protocol definition was updated to match that NEO/py release in
the previous patches.

This corresponds to NEO/py commit 2a27239d (tweak: add option to
simulate).

This corresponds to NEO/py commit c2c9e99d (Better error reporting from
the master to neoctl for denied requests).

This corresponds to NEO/py commit 21190ee7 (Make 'neoctl print pt'
report the number of replicas).

This corresponds to NEO/py commit ef5fc508 (Make the number of replicas
modifiable when the cluster is running).

One important change in the protocol is that Client no longer queries
Master for partition table - instead M pushed partTab to C right after
identification (after pushing nodeTab).

See also: https://neo.nexedi.com/P-NEO-Protocol.Specification.2019?portal_skin=CI_slideshow#/9/5

This corresponds to NEO/py commit 27e3f620 (New --new-nid storage option
for fast cloning).

NEO 1.12

* tag 'v1.12': (28 commits)
  Release version 1.12
  master: reject drop/tweak ctl commands that could lead to unwanted status
  qa: extend test reproducing the migration of a big ZODB to NEO
  neoctl: better display of full partition tables
  Bump protocol version
  tweak: add option to simulate
  tweak: do not crash when trying to remove all nodes
  tweak: do not touch cells of nodes that are intended to be dropped
  Better error reporting from the master to neoctl for denied requests
  Make 'neoctl print pt' report the number of replicas
  Make the number of replicas modifiable when the cluster is running
  New --new-nid storage option for fast cloning
  qa: fix 2 tests with ZODB5
  qa: new tools/stress options to evaluate MySQL engines
  qa: provide a way to let tests start 1 mysqld per storage node
  mysql: make 'user' actually optional in the DB connection string
  mysql: specify column families for RocksDB
  qa: add testIncremental (testImporter) test
  importer: fix hidden "maximum recursion depth exceeded" at startup
  importer: fix closure of ZODB, and also do it when the import is finished
  sqlite: fix resumption of migration to NEO with Importer
  qa: fix a random failure in threaded tests
  importer: speed up startup when the import is already finished
  importer: fix replication (as source) once import is finished
  storage: fix DatabaseManager.getLastTID with max_tid
  qa: remove 2 useless unit tests
  storage: allow the master to change our node id
  Rename --uuid command-line options into --nid
  importer: fix possible data loss on writeback

This protocol version corresponds to protocol version used by NEO/py
v1.11 and was set in NEO/py commit 9a5b46dd (Bump protocol version).

The protocol definition was updated to match that NEO/py release in
the previous patch.

This corresponds to NEO/py commit 64826794 (New neoctl command to flush
the logs of all nodes in the cluster).

NEO 1.11

* tag 'v1.11': (52 commits)
  Release version 1.11
  Fix short descriptions of neoctl & neomigrate in their headers
  Update copyright year
  qa: new tool to stress-test NEO
  master: fix typo in comment
  Fix error handling when setting up a listening connector
  Fix incomplete/incorrect mapping of node ids in logs
  Fix log corruption on rotation in multi-threaded applications (e.g. client)
  sqlite: optimize storage of metadata
  neolog: do not die when a table is corrupted
  neolog: add support for zstd-compressed logs
  neolog: do not hardcode default value of -L option in help message
  fixup! New log format to show node id (and optionally cluster name) in node column
  New log format to show node id (and optionally cluster name) in node column
  fixup! client: discard late answers to lockless writes
  client: fix race condition between Storage.load() and invalidations
  client: fix race condition in refcounting dispatched answer packets
  More RTMIN+2 (log) information for clients and connections
  storage: check for conflicts when notifying that the a partition is replicated
  storage: clarify several assertions
  qa: new expectedFailure testcase method
  client: merge ConnectionPool inside Application
  client: prepare merge of ConnectionPool inside Application
  client: fix AssertionError when trying to reconnect too quickly after an error
  qa: fix attributeTracker
  storage: fix storage leak when an oid is stored several times within a transaction
  client: discard late answers to lockless writes
  qa: in threaded tests, log queued packets when "tic is looping forever"
  In logs, dump the partition table in a more compact and readable way
  storage: fix write-locking bug when a deadlock happens at the end of a replication
  client: log_flush most exceptions raised from Application to ZODB
  client: fix assertion failure in case of conflict + storage disconnection
  client: simplify connection management in transaction contexts
  client: also vote to nodes that only check serials
  qa: deindent code
  Bump protocol version
  client: fix undetected disconnections to storage nodes during commit
  Fix data corruption due to undetected conflicts after storage failures
  master: notify replicating nodes of aborted watched transactions
  New neoctl command to flush the logs of all nodes in the cluster
  storage: fix premature write-locking during rebase when replication ends
  client: fix race condition when a storage connection is closed just after identification
  storage: relax assertion
  comments, unused import
  storage: fix write-lock leak
  client: fix possible corruption in case of network failure with a storage
  qa: comment about potential freeze when a functional test ends
  storage: fix assertion failure in case of connection reset with a client node
  qa: document a rare random failure in testExport
  debug: add script to trace all accesses to the client cache
  Use argparse instead of optparse
  neolog: use argparse instead of optparse
  Add comment about dormant bug when sending a lot of data to a slow node
  client: make clearer that max_size attribute is used from outside ClientCache