Commits · fa50260370770f90c860cf6554677945d295742e · Kirill Smelkov / neo

05 Nov, 2020 9 commits

Kirill Smelkov authored Nov 05, 2020

* master:
  go/zodb/zeo: Correctly prefix zLink-related errors with operational context
  go/{zeo,neo}: handshake: Report "unexpected EOF" if peer sends only partial data
  go/{zeo,neo}: Don't tolerate if a packet reads only partially
  go/zodb/fs1: Factor-out noEOF and okEOF into common place (-> internal/xio)

fa502603

go/zodb/zeo: Correctly prefix zLink-related errors with operational context · 00ace0e8

Kirill Smelkov authored Nov 05, 2020

Go through zrpc and add error context to every function that can create
error. This is similar to what NEO does in neonet. The reason to do this
was the following obscure error from wcfs:

E1105 11:32:33.295497 24639 wcfs.go:2505] zwatch zeo://:28359: EOF

which, after this patch, is now

E1105 12:53:15.922052 30731 wcfs.go:2510] zwatch zeo://:27024: zlink 127.0.0.1:47768 - 127.0.0.1:27024: recvPkt: EOF

00ace0e8

go/{zeo,neo}: handshake: Report "unexpected EOF" if peer sends only partial data · 4d781f60
Kirill Smelkov authored Nov 05, 2020
```
NEO was already doing this, but let's use xio.NoEOF everywhere for
uniformity.
```
4d781f60
go/{zeo,neo}: Don't tolerate if a packet reads only partially · 72aea8ce
Kirill Smelkov authored Nov 05, 2020
```
If remote peer closes the link half-way of packet data - turn EOF into
ErrUnexpectedEOF.
```
72aea8ce
go/zodb/fs1: Factor-out noEOF and okEOF into common place (-> internal/xio) · 2babcd59
Kirill Smelkov authored Nov 05, 2020
```
We will need to use those utilities for ZEO and NEO.
```
2babcd59

go/zodb/zeo: Correctly prefix zLink-related errors with operational context · 2d12ae8c

Kirill Smelkov authored Nov 05, 2020

Go through zrpc and add error context to every function that can create
error. This is similar to what NEO does in neonet. The reason to do this
was the following obscure error from wcfs:

E1105 11:32:33.295497 24639 wcfs.go:2505] zwatch zeo://:28359: EOF

which, after this patch, is now

E1105 12:53:15.922052 30731 wcfs.go:2510] zwatch zeo://:27024: zlink 127.0.0.1:47768 - 127.0.0.1:27024: recvPkt: EOF

2d12ae8c

go/{zeo,neo}: handshake: Report "unexpected EOF" if peer sends only partial data · de9f6f64
Kirill Smelkov authored Nov 05, 2020
```
NEO was already doing this, but let's use xio.NoEOF everywhere for
uniformity.
```
de9f6f64
go/{zeo,neo}: Don't tolerate if a packet reads only partially · 1986a126
Kirill Smelkov authored Nov 05, 2020
```
If remote peer closes the link half-way of packet data - turn EOF into
ErrUnexpectedEOF.
```
1986a126
go/zodb/fs1: Factor-out noEOF and okEOF into common place (-> internal/xio) · b664d749
Kirill Smelkov authored Nov 05, 2020
```
We will need to use those utilities for ZEO and NEO.
```
b664d749

04 Nov, 2020 10 commits

Merge branch 'master' into t · 58689cb6

Kirill Smelkov authored Nov 04, 2020

* master:
  go/neo/neonet: Increase signal/noise in dump of packets with decode problems
  fixup! go/neo/proto: Switch enum encoding from int32 to int8
  go/zodb/zeo: Load: loadBefore RPC can return None to indicate non-existing object
  go/internal/xtesting: DrvTestLoad: Verify load of non-existing and not-yet-created objects

58689cb6

go/neo/neonet: Increase signal/noise in dump of packets with decode problems · 774f12c3

Kirill Smelkov authored Nov 04, 2020

We recently hit "decode: buffer overflow" errors due to mismatch in
between NEO/go and NEO/py (see previous patch). With dumpio=true that
problem was showing itself as

    127.0.0.1:50624 > 127.0.0.1:41863: .5 GetObject &{0000000000000000 ffffffffffffffff 0285cbac258bf266}
    127.0.0.1:50624 < 127.0.0.1:41863: .5 (proto.Error) decode: buffer overflow; #24 [24]: 00 00 00 03 00 00 00 10 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 Error ACK ;  [24]tail: 00 00 00 03 00 00 00 10 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30
        xtesting.go:306: load 0285cbac258bf265:0000000000000000: returned err unexpected:
            have: neo://1@127.0.0.1:24078: load 0285cbac258bf265:0000000000000000: 127.0.0.1:50624 - 127.0.0.1:41863 .5: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:24078: load 0285cbac258bf265:0000000000000000: 0000000000000000: object was not yet created

Here after printing error and dumping all bytes from packet payload, it
was printing again message type, message value from to-be-decoded place
(which is zero-initialized) and data bytes again.
-> Fix it not to print anything after dump of payload data:

    127.0.0.1:60518 > 127.0.0.1:46719: .5 GetObject &{0000000000000000 ffffffffffffffff 0285cbac258bf266}
    127.0.0.1:60518 < 127.0.0.1:46719: .5 (Error) decode: buffer overflow; #24 [24]: 00 00 00 03 00 00 00 10 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30
        xtesting.go:306: load 0285cbac258bf265:0000000000000000: returned err unexpected:
            have: neo://1@127.0.0.1:27853: load 0285cbac258bf265:0000000000000000: 127.0.0.1:60518 - 127.0.0.1:46719 .5: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:27853: load 0285cbac258bf265:0000000000000000: 0000000000000000: object was not yet created

774f12c3

fixup! go/neo/proto: Switch enum encoding from int32 to int8 · 05463172

Kirill Smelkov authored Nov 04, 2020

In commit 5f13cc85 I changed enum encodings from int32 to int8, but did
not noticed that NEO/py commit 52db5607 ("protocol: a single byte is
more than enough to encode enums") despite specified intent and
ErrorCodes being marked with @Enum, changed encoding only for fields
that are marked as PEnum in structures. In NEO/py the Error.code field
is still marked as PNumber which encodes as 32-bit integer on the wire.
-> Fix it back.

With recent xtesting.DrvTestLoad update this error was manifesting itself as (on @t branch):

--- FAIL: TestLoad (2.08s)
    --- FAIL: TestLoad/py (2.07s)
        xtesting.go:306: load 0285cbac258bf265:0000000000000000: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbac258bf265:0000000000000000: 127.0.0.1:42288 - 127.0.0.1:37109 .5: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbac258bf265:0000000000000000: 0000000000000000: object was not yet created
        xtesting.go:306: load 0285cbac3d0369e5:0000000000000001: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbac3d0369e5:0000000000000001: 127.0.0.1:42288 - 127.0.0.1:37109 .13: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbac3d0369e5:0000000000000001: 0000000000000001: object was not yet created
        xtesting.go:306: load 0285cbac41b4e832:0000000000000002: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbac41b4e832:0000000000000002: 127.0.0.1:42288 - 127.0.0.1:37109 .21: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbac41b4e832:0000000000000002: 0000000000000002: object was not yet created
        xtesting.go:306: load 0285cbac4666667f:0000000000000003: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbac4666667f:0000000000000003: 127.0.0.1:42288 - 127.0.0.1:37109 .29: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbac4666667f:0000000000000003: 0000000000000003: object was not yet created
        xtesting.go:306: load 0285cbac4fc96318:0000000000000004: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbac4fc96318:0000000000000004: 127.0.0.1:42288 - 127.0.0.1:37109 .41: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbac4fc96318:0000000000000004: 0000000000000004: object was not yet created
        xtesting.go:306: load 0285cbac547ae165:0000000000000005: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbac547ae165:0000000000000005: 127.0.0.1:42288 - 127.0.0.1:37109 .49: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbac547ae165:0000000000000005: 0000000000000005: object was not yet created
        xtesting.go:306: load 0285cbac628f5c4b:0000000000000006: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbac628f5c4b:0000000000000006: 127.0.0.1:42288 - 127.0.0.1:37109 .65: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbac628f5c4b:0000000000000006: 0000000000000006: object was not yet created
        xtesting.go:306: load 0285cbaca444447f:0000000000000007: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbaca444447f:0000000000000007: 127.0.0.1:42288 - 127.0.0.1:37109 .125: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbaca444447f:0000000000000007: 0000000000000007: object was not yet created
        xtesting.go:306: load 0285cbacbbbbbbff:0000000000000008: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbacbbbbbbff:0000000000000008: 127.0.0.1:42288 - 127.0.0.1:37109 .149: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbacbbbbbbff:0000000000000008: 0000000000000008: object was not yet created
        xtesting.go:306: load 0285cbad80da7498:0000000000000009: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbad80da7498:0000000000000009: 127.0.0.1:42288 - 127.0.0.1:37109 .269: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbad80da7498:0000000000000009: 0000000000000009: object was not yet created
        xtesting.go:331: load 7fffffffffffffff:000000000000000a: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 7fffffffffffffff:000000000000000a: 127.0.0.1:42288 - 127.0.0.1:37109 .295: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 7fffffffffffffff:000000000000000a: 000000000000000a: no such object

05463172

go/zodb/zeo: Load: loadBefore RPC can return None to indicate non-existing object · f8696942

Kirill Smelkov authored Nov 04, 2020

Before this patch and with updated DrvTestLoad (see previous patch) it was failing as:

--- FAIL: TestLoad (0.52s)
--- FAIL: TestLoad/py/msgpack=false (0.25s)
xtesting.go:306: load 0285cbac258bf265:0000000000000000: returned err unexpected:
have: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac258bf265:0000000000000000: /tmp/zeo535364855/1.fs.zeosock: call loadBefore: unexpected reply: got ogórek.None{}; expect 3-tuple
want: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac258bf265:0000000000000000: 0000000000000000: no such object
xtesting.go:306: load 0285cbac3d0369e5:0000000000000001: returned err unexpected:
have: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac3d0369e5:0000000000000001: /tmp/zeo535364855/1.fs.zeosock: call loadBefore: unexpected reply: got ogórek.None{}; expect 3-tuple
want: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac3d0369e5:0000000000000001: 0000000000000001: no such object
xtesting.go:306: load 0285cbac41b4e832:0000000000000002: returned err unexpected:
have: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac41b4e832:0000000000000002: /tmp/zeo535364855/1.fs.zeosock: call loadBefore: unexpected reply: got ogórek.None{}; expect 3-tuple
want: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac41b4e832:0000000000000002: 0000000000000002: no such object
xtesting.go:306: load 0285cbac4666667f:0000000000000003: returned err unexpected:
have: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac4666667f:0000000000000003: /tmp/zeo535364855/1.fs.zeosock: call loadBefore: unexpected reply: got ogórek.None{}; expect 3-tuple
want: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac4666667f:0000000000000003: 0000000000000003: no such object
xtesting.go:306: load 0285cbac4fc96318:0000000000000004: returned err unexpected:
have: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac4fc96318:0000000000000004: /tmp/zeo535364855/1.fs.zeosock: call loadBefore: unexpected reply: got ogórek.None{}; expect 3-tuple
want: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac4fc96318:0000000000000004: 0000000000000004: no such object
xtesting.go:306: load 0285cbac547ae165:0000000000000005: returned err unexpected:
have: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac547ae165:0000000000000005: /tmp/zeo535364855/1.fs.zeosock: call loadBefore: unexpected reply: got ogórek.None{}; expect 3-tuple
want: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac547ae165:0000000000000005: 0000000000000005: no such object
xtesting.go:306: load 0285cbac628f5c4b:0000000000000006: returned err unexpected:
have: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac628f5c4b:0000000000000006: /tmp/zeo535364855/1.fs.zeosock: call loadBefore: unexpected reply: got ogórek.None{}; expect 3-tuple
want: /tmp/zeo535364855/1.fs.zeosock: load 0285cbac628f5c4b:0000000000000006: 0000000000000006: no such object
xtesting.go:306: load 0285cbaca444447f:0000000000000007: returned err unexpected:
have: /tmp/zeo535364855/1.fs.zeosock: load 0285cbaca444447f:0000000000000007: /tmp/zeo535364855/1.fs.zeosock: call loadBefore: unexpected reply: got ogórek.None{}; expect 3-tuple
want: /tmp/zeo535364855/1.fs.zeosock: load 0285cbaca444447f:0000000000000007: 0000000000000007: no such object
xtesting.go:306: load 0285cbacbbbbbbff:0000000000000008: returned err unexpected:
have: /tmp/zeo535364855/1.fs.zeosock: load 0285cbacbbbbbbff:0000000000000008: /tmp/zeo535364855/1.fs.zeosock: call loadBefore: unexpected reply: got ogórek.None{}; expect 3-tuple
want: /tmp/zeo535364855/1.fs.zeosock: load 0285cbacbbbbbbff:0000000000000008: 0000000000000008: no such object
xtesting.go:306: load 0285cbad80da7498:0000000000000009: returned err unexpected:
have: /tmp/zeo535364855/1.fs.zeosock: load 0285cbad80da7498:0000000000000009: /tmp/zeo535364855/1.fs.zeosock: call loadBefore: unexpected reply: got ogórek.None{}; expect 3-tuple
want: /tmp/zeo535364855/1.fs.zeosock: load 0285cbad80da7498:0000000000000009: 0000000000000009: no such object
--- FAIL: TestLoad/py/msgpack=true (0.26s)
xtesting.go:306: load 0285cbac258bf265:0000000000000000: returned err unexpected:
have: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac258bf265:0000000000000000: /tmp/zeo794664426/1.fs.zeosock: call loadBefore: zlink is closed
want: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac258bf265:0000000000000000: 0000000000000000: no such object
xtesting.go:306: load 0285cbac3d0369e5:0000000000000001: returned err unexpected:
have: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac3d0369e5:0000000000000001: /tmp/zeo794664426/1.fs.zeosock: call loadBefore: zlink is closed
want: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac3d0369e5:0000000000000001: 0000000000000001: no such object
xtesting.go:306: load 0285cbac41b4e832:0000000000000002: returned err unexpected:
have: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac41b4e832:0000000000000002: /tmp/zeo794664426/1.fs.zeosock: call loadBefore: zlink is closed
want: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac41b4e832:0000000000000002: 0000000000000002: no such object
xtesting.go:306: load 0285cbac4666667f:0000000000000003: returned err unexpected:
have: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac4666667f:0000000000000003: /tmp/zeo794664426/1.fs.zeosock: call loadBefore: zlink is closed
want: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac4666667f:0000000000000003: 0000000000000003: no such object
xtesting.go:306: load 0285cbac4fc96318:0000000000000004: returned err unexpected:
have: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac4fc96318:0000000000000004: /tmp/zeo794664426/1.fs.zeosock: call loadBefore: zlink is closed
want: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac4fc96318:0000000000000004: 0000000000000004: no such object
xtesting.go:306: load 0285cbac547ae165:0000000000000005: returned err unexpected:
have: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac547ae165:0000000000000005: /tmp/zeo794664426/1.fs.zeosock: call loadBefore: zlink is closed
want: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac547ae165:0000000000000005: 0000000000000005: no such object
xtesting.go:306: load 0285cbac628f5c4b:0000000000000006: returned err unexpected:
have: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac628f5c4b:0000000000000006: /tmp/zeo794664426/1.fs.zeosock: call loadBefore: zlink is closed
want: /tmp/zeo794664426/1.fs.zeosock: load 0285cbac628f5c4b:0000000000000006: 0000000000000006: no such object
xtesting.go:306: load 0285cbaca444447f:0000000000000007: returned err unexpected:
have: /tmp/zeo794664426/1.fs.zeosock: load 0285cbaca444447f:0000000000000007: /tmp/zeo794664426/1.fs.zeosock: call loadBefore: zlink is closed
want: /tmp/zeo794664426/1.fs.zeosock: load 0285cbaca444447f:0000000000000007: 0000000000000007: no such object
xtesting.go:306: load 0285cbacbbbbbbff:0000000000000008: returned err unexpected:
have: /tmp/zeo794664426/1.fs.zeosock: load 0285cbacbbbbbbff:0000000000000008: /tmp/zeo794664426/1.fs.zeosock: call loadBefore: zlink is closed
want: /tmp/zeo794664426/1.fs.zeosock: load 0285cbacbbbbbbff:0000000000000008: 0000000000000008: no such object
xtesting.go:306: load 0285cbad80da7498:0000000000000009: returned err unexpected:
have: /tmp/zeo794664426/1.fs.zeosock: load 0285cbad80da7498:0000000000000009: /tmp/zeo794664426/1.fs.zeosock: call loadBefore: zlink is closed
want: /tmp/zeo794664426/1.fs.zeosock: load 0285cbad80da7498:0000000000000009: 0000000000000009: no such object

f8696942

go/internal/xtesting: DrvTestLoad: Verify load of non-existing and not-yet-created objects · 4acadb94
Kirill Smelkov authored Nov 04, 2020
```
This currently fails on ZEO and NEO. Those storages will be fixed in the
following patches (NEO only on @t branch for now).
```
4acadb94

go/neo/proto: Don't require deletedAt to be present in OID_NOT_FOUND · a2b8516c

Kirill Smelkov authored Nov 04, 2020

Providing deletedAt in addition to oid in OID_NOT_FOUND error is NEO/go extension.
NEO/py sends only OidNotFound(oid):

https://lab.nexedi.com/nexedi/neoppod/blob/v1.12-0-g6332112c/neo/storage/handlers/client.py#L70-72

-> decode deletedAt only if it is present.

Since inspecting NEO/py codebase turned out that message always comes
with oid, remove the "abusing" words about this approach to convey oid
via error message for OID_NOT_FOUND and OID_DOES_NOT_EXIST.

Without this patch TestLoad was failing as

    === RUN   TestLoad/py
    I: runneo.py: /tmp/neo124851059/1: started master(s): 127.0.0.1:19681
        xtesting.go:306: load 0285cbac258bf265:0000000000000000: returned err unexpected:
            have: neo://1@127.0.0.1:19681: load 0285cbac258bf265:0000000000000000: OID_NOT_FOUND: 0000000000000000
            want: neo://1@127.0.0.1:19681: load 0285cbac258bf265:0000000000000000: 0000000000000000: object was not yet created
        xtesting.go:306: load 0285cbac3d0369e5:0000000000000001: returned err unexpected:
            have: neo://1@127.0.0.1:19681: load 0285cbac3d0369e5:0000000000000001: OID_NOT_FOUND: 0000000000000001
            want: neo://1@127.0.0.1:19681: load 0285cbac3d0369e5:0000000000000001: 0000000000000001: object was not yet created
        xtesting.go:306: load 0285cbac41b4e832:0000000000000002: returned err unexpected:
            have: neo://1@127.0.0.1:19681: load 0285cbac41b4e832:0000000000000002: OID_NOT_FOUND: 0000000000000002
            want: neo://1@127.0.0.1:19681: load 0285cbac41b4e832:0000000000000002: 0000000000000002: object was not yet created
        xtesting.go:306: load 0285cbac4666667f:0000000000000003: returned err unexpected:
            have: neo://1@127.0.0.1:19681: load 0285cbac4666667f:0000000000000003: OID_NOT_FOUND: 0000000000000003
            want: neo://1@127.0.0.1:19681: load 0285cbac4666667f:0000000000000003: 0000000000000003: object was not yet created
        xtesting.go:306: load 0285cbac4fc96318:0000000000000004: returned err unexpected:
            have: neo://1@127.0.0.1:19681: load 0285cbac4fc96318:0000000000000004: OID_NOT_FOUND: 0000000000000004
            want: neo://1@127.0.0.1:19681: load 0285cbac4fc96318:0000000000000004: 0000000000000004: object was not yet created
        xtesting.go:306: load 0285cbac547ae165:0000000000000005: returned err unexpected:
            have: neo://1@127.0.0.1:19681: load 0285cbac547ae165:0000000000000005: OID_NOT_FOUND: 0000000000000005
            want: neo://1@127.0.0.1:19681: load 0285cbac547ae165:0000000000000005: 0000000000000005: object was not yet created
        xtesting.go:306: load 0285cbac628f5c4b:0000000000000006: returned err unexpected:
            have: neo://1@127.0.0.1:19681: load 0285cbac628f5c4b:0000000000000006: OID_NOT_FOUND: 0000000000000006
            want: neo://1@127.0.0.1:19681: load 0285cbac628f5c4b:0000000000000006: 0000000000000006: object was not yet created
        xtesting.go:306: load 0285cbaca444447f:0000000000000007: returned err unexpected:
            have: neo://1@127.0.0.1:19681: load 0285cbaca444447f:0000000000000007: OID_NOT_FOUND: 0000000000000007
            want: neo://1@127.0.0.1:19681: load 0285cbaca444447f:0000000000000007: 0000000000000007: object was not yet created
        xtesting.go:306: load 0285cbacbbbbbbff:0000000000000008: returned err unexpected:
            have: neo://1@127.0.0.1:19681: load 0285cbacbbbbbbff:0000000000000008: OID_NOT_FOUND: 0000000000000008
            want: neo://1@127.0.0.1:19681: load 0285cbacbbbbbbff:0000000000000008: 0000000000000008: object was not yet created
        xtesting.go:306: load 0285cbad80da7498:0000000000000009: returned err unexpected:
            have: neo://1@127.0.0.1:19681: load 0285cbad80da7498:0000000000000009: OID_NOT_FOUND: 0000000000000009
            want: neo://1@127.0.0.1:19681: load 0285cbad80da7498:0000000000000009: 0000000000000009: object was not yet created

a2b8516c

go/neo/neonet: Increase signal/noise in dump of packets with decode problems · dacce3d1

Kirill Smelkov authored Nov 04, 2020

We recently hit "decode: buffer overflow" errors due to mismatch in
between NEO/go and NEO/py (see previous patch). With dumpio=true that
problem was showing itself as

    127.0.0.1:50624 > 127.0.0.1:41863: .5 GetObject &{0000000000000000 ffffffffffffffff 0285cbac258bf266}
    127.0.0.1:50624 < 127.0.0.1:41863: .5 (proto.Error) decode: buffer overflow; #24 [24]: 00 00 00 03 00 00 00 10 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 Error ACK ;  [24]tail: 00 00 00 03 00 00 00 10 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30
        xtesting.go:306: load 0285cbac258bf265:0000000000000000: returned err unexpected:
            have: neo://1@127.0.0.1:24078: load 0285cbac258bf265:0000000000000000: 127.0.0.1:50624 - 127.0.0.1:41863 .5: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:24078: load 0285cbac258bf265:0000000000000000: 0000000000000000: object was not yet created

Here after printing error and dumping all bytes from packet payload, it
was printing again message type, message value from to-be-decoded place
(which is zero-initialized) and data bytes again.
-> Fix it not to print anything after dump of payload data:

    127.0.0.1:60518 > 127.0.0.1:46719: .5 GetObject &{0000000000000000 ffffffffffffffff 0285cbac258bf266}
    127.0.0.1:60518 < 127.0.0.1:46719: .5 (Error) decode: buffer overflow; #24 [24]: 00 00 00 03 00 00 00 10 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30
        xtesting.go:306: load 0285cbac258bf265:0000000000000000: returned err unexpected:
            have: neo://1@127.0.0.1:27853: load 0285cbac258bf265:0000000000000000: 127.0.0.1:60518 - 127.0.0.1:46719 .5: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:27853: load 0285cbac258bf265:0000000000000000: 0000000000000000: object was not yet created

dacce3d1

fixup! go/neo/proto: Switch enum encoding from int32 to int8 · 52cd5b4f

Kirill Smelkov authored Nov 04, 2020

In commit 144dafa0 I changed enum encodings from int32 to int8, but did
not noticed that NEO/py commit 52db5607 ("protocol: a single byte is
more than enough to encode enums") despite specified intent and
ErrorCodes being marked with @Enum, changed encoding only for fields
that are marked as PEnum in structures. In NEO/py the Error.code field
is still marked as PNumber which encodes as 32-bit integer on the wire.
-> Fix it back.

With recent xtesting.DrvTestLoad update this error was manifesting itself as

--- FAIL: TestLoad (2.08s)
    --- FAIL: TestLoad/py (2.07s)
        xtesting.go:306: load 0285cbac258bf265:0000000000000000: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbac258bf265:0000000000000000: 127.0.0.1:42288 - 127.0.0.1:37109 .5: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbac258bf265:0000000000000000: 0000000000000000: object was not yet created
        xtesting.go:306: load 0285cbac3d0369e5:0000000000000001: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbac3d0369e5:0000000000000001: 127.0.0.1:42288 - 127.0.0.1:37109 .13: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbac3d0369e5:0000000000000001: 0000000000000001: object was not yet created
        xtesting.go:306: load 0285cbac41b4e832:0000000000000002: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbac41b4e832:0000000000000002: 127.0.0.1:42288 - 127.0.0.1:37109 .21: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbac41b4e832:0000000000000002: 0000000000000002: object was not yet created
        xtesting.go:306: load 0285cbac4666667f:0000000000000003: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbac4666667f:0000000000000003: 127.0.0.1:42288 - 127.0.0.1:37109 .29: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbac4666667f:0000000000000003: 0000000000000003: object was not yet created
        xtesting.go:306: load 0285cbac4fc96318:0000000000000004: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbac4fc96318:0000000000000004: 127.0.0.1:42288 - 127.0.0.1:37109 .41: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbac4fc96318:0000000000000004: 0000000000000004: object was not yet created
        xtesting.go:306: load 0285cbac547ae165:0000000000000005: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbac547ae165:0000000000000005: 127.0.0.1:42288 - 127.0.0.1:37109 .49: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbac547ae165:0000000000000005: 0000000000000005: object was not yet created
        xtesting.go:306: load 0285cbac628f5c4b:0000000000000006: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbac628f5c4b:0000000000000006: 127.0.0.1:42288 - 127.0.0.1:37109 .65: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbac628f5c4b:0000000000000006: 0000000000000006: object was not yet created
        xtesting.go:306: load 0285cbaca444447f:0000000000000007: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbaca444447f:0000000000000007: 127.0.0.1:42288 - 127.0.0.1:37109 .125: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbaca444447f:0000000000000007: 0000000000000007: object was not yet created
        xtesting.go:306: load 0285cbacbbbbbbff:0000000000000008: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbacbbbbbbff:0000000000000008: 127.0.0.1:42288 - 127.0.0.1:37109 .149: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbacbbbbbbff:0000000000000008: 0000000000000008: object was not yet created
        xtesting.go:306: load 0285cbad80da7498:0000000000000009: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 0285cbad80da7498:0000000000000009: 127.0.0.1:42288 - 127.0.0.1:37109 .269: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 0285cbad80da7498:0000000000000009: 0000000000000009: object was not yet created
        xtesting.go:331: load 7fffffffffffffff:000000000000000a: returned err unexpected:
            have: neo://1@127.0.0.1:32731: load 7fffffffffffffff:000000000000000a: 127.0.0.1:42288 - 127.0.0.1:37109 .295: decode: decode: buffer overflow
            want: neo://1@127.0.0.1:32731: load 7fffffffffffffff:000000000000000a: 000000000000000a: no such object

52cd5b4f

go/zodb/zeo: Load: loadBefore RPC can return None to indicate non-existing object · 080d5450

Kirill Smelkov authored Nov 04, 2020

Before this patch and with updated DrvTestLoad (see previous patch) it was failing as:

080d5450

go/internal/xtesting: DrvTestLoad: Verify load of non-existing and not-yet-created objects · 523298bc
Kirill Smelkov authored Nov 04, 2020
```
This currently fails on ZEO and NEO. Those storages will be fixed in the
following patches.
```
523298bc

03 Nov, 2020 6 commits

. · eb8d8682
Kirill Smelkov authored Nov 03, 2020

eb8d8682
X py: Don't import MySQLdb where it is not needed · d84749a8
Kirill Smelkov authored Nov 03, 2020

d84749a8

Merge branch 'master' into t · 3e7dfb6a

Kirill Smelkov authored Nov 03, 2020

* master:
  neotest: test-go: Disable test results caching
  go/neo/t: nxd/runTestSuite was generalized and factored-out into nxdtest

3e7dfb6a

. · 2d1d3ebd
Kirill Smelkov authored Nov 03, 2020

2d1d3ebd

neotest: test-go: Disable test results caching · d8b51133

Kirill Smelkov authored Nov 03, 2020

Starting from Go1.10 `go test` caches successful test results[1].
However we want to torture the implementation and run the tests for real
each time `neotest test-go` is ran.

-> Disable tests caching

[1] https://golang.org/doc/go1.10#test

d8b51133

go/neo/t: nxd/runTestSuite was generalized and factored-out into nxdtest · 19ff426a

Kirill Smelkov authored Nov 03, 2020

See:

- https://lab.nexedi.com/nexedi/nxdtest
- https://lab.nexedi.com/nexedi/nxdtest/blob/master/nxdtest/__init__.py
- nexedi/nxdtest@d575236a
- nexedi/slapos!839

-> Leave only .nxdtest file with neotest-specific bits to be run by
nxdtest driver.

19ff426a

01 Nov, 2020 15 commits

Merge branch 'master' into t · a179fa40
Kirill Smelkov authored Nov 01, 2020
```
* master:
```
a179fa40

Sync with NEO/py v1.12-13-g (oldproto branch) · d14040ac

Kirill Smelkov authored Nov 01, 2020

* origin/old-proto:
  qa: skip broken ZODB test
  client: fix race with invalidations when starting a new transaction on ZODB 5
  Code clean-up, comment fixes
  master: fix crash in STARTING_BACKUP when connecting to an upstream secondary master
  mysql: workaround for MDEV-20693
  client: inline Application._loadFromCache
  client: replace global load lock by a per-oid one
  client: unindent code
  client: remove load lock in tpc_finish
  qa: check cache in testExternalInvalidation
  qa: comment testExternalInvalidation2

d14040ac

Sync with NEO/py v1.12-13-g (oldproto branch) · 3a6a3475

Kirill Smelkov authored Nov 01, 2020

* origin/old-proto:
  qa: skip broken ZODB test
  client: fix race with invalidations when starting a new transaction on ZODB 5
  Code clean-up, comment fixes
  master: fix crash in STARTING_BACKUP when connecting to an upstream secondary master
  mysql: workaround for MDEV-20693
  client: inline Application._loadFromCache
  client: replace global load lock by a per-oid one
  client: unindent code
  client: remove load lock in tpc_finish
  qa: check cache in testExternalInvalidation
  qa: comment testExternalInvalidation2

3a6a3475

Merge branch 'master' into t · a4bf053e

Kirill Smelkov authored Nov 01, 2020

* master:
  go/neo/proto: Version 5 -> 6
  go/neo/proto: tweak += dry-run
  go/neo/proto: ErrorCode += DENIED
  go/neo/proto: AnswerPartitionList += NumReplicas
  go/neo/proto: Sync with NEO/py on changes to make the number of replicas modifiable at runtime
  go/neo/proto: RequestIdentification += NewNID; Move .DevPath

a4bf053e

Sync with NEO/py v1.12 · 8ea85c71
Kirill Smelkov authored Nov 01, 2020

8ea85c71

go/neo/proto: Version 5 -> 6 · d4c2253f

Kirill Smelkov authored Nov 01, 2020

This protocol version corresponds to protocol version used by NEO/py
v1.12 and was set in NEO/py commit c6453626 (Bump protocol version).

The protocol definition was updated to match that NEO/py release in
the previous patches.

d4c2253f

go/neo/proto: tweak += dry-run · 595052c2
Kirill Smelkov authored Nov 01, 2020
```
This corresponds to NEO/py commit 2a27239d (tweak: add option to
simulate).
```
595052c2

go/neo/proto: ErrorCode += DENIED · 484c34a5

Kirill Smelkov authored Nov 01, 2020

This corresponds to NEO/py commit c2c9e99d (Better error reporting from
the master to neoctl for denied requests).

484c34a5

go/neo/proto: AnswerPartitionList += NumReplicas · 56835167
Kirill Smelkov authored Nov 01, 2020
```
This corresponds to NEO/py commit 21190ee7 (Make 'neoctl print pt'
report the number of replicas).
```
56835167

go/neo/proto: Sync with NEO/py on changes to make the number of replicas modifiable at runtime · 0d75e144

Kirill Smelkov authored Nov 01, 2020

This corresponds to NEO/py commit ef5fc508 (Make the number of replicas
modifiable when the cluster is running).

One important change in the protocol is that Client no longer queries
Master for partition table - instead M pushed partTab to C right after
identification (after pushing nodeTab).

See also: https://neo.nexedi.com/P-NEO-Protocol.Specification.2019?portal_skin=CI_slideshow#/9/5

0d75e144

go/neo/proto: RequestIdentification += NewNID; Move .DevPath · 98869a9b
Kirill Smelkov authored Nov 01, 2020
```
This corresponds to NEO/py commit 27e3f620 (New --new-nid storage option
for fast cloning).
```
98869a9b

Merge tag v1.12 · c85756a0

Kirill Smelkov authored Nov 01, 2020

NEO 1.12

* tag 'v1.12': (28 commits)
  Release version 1.12
  master: reject drop/tweak ctl commands that could lead to unwanted status
  qa: extend test reproducing the migration of a big ZODB to NEO
  neoctl: better display of full partition tables
  Bump protocol version
  tweak: add option to simulate
  tweak: do not crash when trying to remove all nodes
  tweak: do not touch cells of nodes that are intended to be dropped
  Better error reporting from the master to neoctl for denied requests
  Make 'neoctl print pt' report the number of replicas
  Make the number of replicas modifiable when the cluster is running
  New --new-nid storage option for fast cloning
  qa: fix 2 tests with ZODB5
  qa: new tools/stress options to evaluate MySQL engines
  qa: provide a way to let tests start 1 mysqld per storage node
  mysql: make 'user' actually optional in the DB connection string
  mysql: specify column families for RocksDB
  qa: add testIncremental (testImporter) test
  importer: fix hidden "maximum recursion depth exceeded" at startup
  importer: fix closure of ZODB, and also do it when the import is finished
  sqlite: fix resumption of migration to NEO with Importer
  qa: fix a random failure in threaded tests
  importer: speed up startup when the import is already finished
  importer: fix replication (as source) once import is finished
  storage: fix DatabaseManager.getLastTID with max_tid
  qa: remove 2 useless unit tests
  storage: allow the master to change our node id
  Rename --uuid command-line options into --nid
  importer: fix possible data loss on writeback

c85756a0

Sync with NEO/py v1.12 · f4b87d77
Kirill Smelkov authored Nov 01, 2020

f4b87d77

go/neo/proto: Version 5 -> 6 · 837f49d1

Kirill Smelkov authored Nov 01, 2020

This protocol version corresponds to protocol version used by NEO/py
v1.12 and was set in NEO/py commit c6453626 (Bump protocol version).

The protocol definition was updated to match that NEO/py release in
the previous patches.

837f49d1

go/neo/proto: tweak += dry-run · 20cacd14
Kirill Smelkov authored Nov 01, 2020
```
This corresponds to NEO/py commit 2a27239d (tweak: add option to
simulate).
```
20cacd14