go get: upgraded github.com/fsnotify/fsnotify v1.4.10-0.20200417215612-7f4cf4dd2b52 => v1.5.1
go get: upgraded github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b => v1.0.0
go get: upgraded github.com/gwenn/gosqlite v0.0.0-20201008200117-82e079acf5b6 => v0.0.0-20211101095637-b18efb2e44c8
go get: upgraded github.com/gwenn/yacr v0.0.0-20200112083327-bbe82c1f4d60 => v0.0.0-20211101095056-492fb0c571bc
go get: upgraded github.com/kisielk/og-rek v1.1.0 => v1.2.0
go get: upgraded github.com/soheilhy/cmux v0.1.5-0.20210205191134-5ec6847320e5 => v0.1.5
go get: upgraded github.com/tinylib/msgp v1.1.5 => v1.1.6
go get: upgraded golang.org/x/net v0.0.0-20210226172049-e18ecbb05110 => v0.0.0-20211111160137-58aab5ef257a
go get: upgraded golang.org/x/sys v0.0.0-20210301091718-77cc2087c03b => v0.0.0-20211111213525-f221eed1c01e
go get: upgraded golang.org/x/text v0.3.5 => v0.3.7
go get: upgraded lab.nexedi.com/kirr/go123 v0.0.0-20210128150852-c20e95f0f789 => v0.0.0-20210906140734-c9eb28d9e408

Highlights:

- og-rek brings support for pickle protocol 5
- go123  brings in support for Go1.17

* master:
  go/zodb/btree: Change V<op> family to also provide visited node key coverage on visit callback
  go/zodb/btree: Add KeyRange type
  go/zodb/btree: Introduce constants for min/max key value
  go/zodb/btree: tests: Don't forget to close storage
  go/zodb/btree: Cosmetics

WCFS needs to know key coverage for every visited node.

WARNING: this is API change.

KeyRange represents [lo,hi) key range. It simplifies working with ranges
of keys. We will use it in the next commit.

KeyRange originated in WCFS and was copied from there:

https://lab.nexedi.com/kirr/wendelin.core/blob/57be0126/wcfs/internal/xbtree/blib/keyrange.go

We were already using math.Min<Key> in one place, but the number of such
places is going to increase. -> Keep min/max definition in only one place.

NOTE: db was already being closed in the test's code.

* t+btree:
  .
  .
  .
  .
  .
  .
  .
  X Start reworking BTree to provide keycov on visit callback

Due to security concerns, go1.17, even though very reluctantly,
similarly to Python broke backward compatibility and stopped to treat ';'
as separator when parsing URL queries.

We still want to accept ';' as separator in credential part.
-> Work it around by replacing ';' -> to '&'.

See https://golang.org/doc/go1.17#semicolons and https://github.com/golang/go/issues/25192 for context.

See also: cf685fb5 (fixup! Y client: Fix URI scheme to move credentials out of query)

parse_qsl no longer treats ';' as valid query separator for security
reason because most proxies did not do so and it was possible to poison
proxy cache due to difference in query separator handling (see bugs.python.org/issue42967).

To handle credentials we don't have any proxy here, and it is still
perfectly valid to use ';' as credentials separator.

-> Fix it with ';' -> '&' replace workaround, before feeding credentials
string to parse_qsl.

Amends: b9a42957.

* master:
  go/internal/xtesting: Add missing X
  go/zodb/{fs1,zeo}: ~staticcheck
  go/zodb/btree: Fix missing return on data-consistency error
  go/zodb, go/zodb/btree: Fix go generate after rename on zodbtools side

staticcheck says:

	xtesting.go:386:2: this value of err is never used (SA4006)

fs1/format.go:204:2: only the first constant in this group has an explicit type (SA9004)
zeo/proto.go:56:2: only the first constant in this group has an explicit type (SA9004)

staticcheck reports

    ziobtree.go:606:4: Errorf is a pure function but its return value is ignored (SA4017)
    ziobtree.go:626:4: Errorf is a pure function but its return value is ignored (SA4017)
    zlobtree.go:606:4: Errorf is a pure function but its return value is ignored (SA4017)
    zlobtree.go:626:4: Errorf is a pure function but its return value is ignored (SA4017)

run_with_zodb3py2_compat was renamed to run_with_zodb4py2_compat in
nexedi/zodbtools@c59a54ca .

Without the fix go genrate was failing as

    (neo) (z-dev) (g.env) kirr@deca:~/src/neo/src/lab.nexedi.com/kirr/neo/go/zodb/btree$ go generate
    Traceback (most recent call last):
      File "./testdata/gen-testdata", line 26, in <module>
        from zodbtools.test.gen_testdata import run_with_zodb3py2_compat
    ImportError: cannot import name run_with_zodb3py2_compat

and

    (neo) (z-dev) (g.env) kirr@deca:~/src/neo/src/lab.nexedi.com/kirr/neo/go/zodb$ go generate
    Traceback (most recent call last):
      File "./py/pydata-gen-testdata", line 24, in <module>
        from zodbtools.test.gen_testdata import run_with_zodb3py2_compat
    ImportError: cannot import name run_with_zodb3py2_compat

This amends commit fc69e00d (go/zodb/fs1: Fix Python database generator
to work with recent zodbtools).

* master:
  go/zodb: Fix ΔTail.LastRevOf based on WCFS experience

- the only valid range for at is [tail, head]. Don't try to return anything
  meaningful for queries outside of this range and just panic instead. This is
  consistent with SliceByRev, which also panics on invalid query, and it is also
  consistent with semantic model that ΔTail is a vector with data keyed by tid in
  range (tail, head]: if key is out of vector range, access to the vector should
  panic, isn't it?

- instead of returning revision of minimum entry on exact=n, always
  return (tail, exact=n) in that case.

The change in behaviour is consistent with ΔFtail and ΔBtail from WCFS
and is needed for ΔFtail to function correctly:

https://lab.nexedi.com/kirr/wendelin.core/blob/22f5f096/wcfs/internal/xbtree/δbtail.go
https://lab.nexedi.com/kirr/wendelin.core/blob/22f5f096/wcfs/internal/zdata/δftail.go

* master:
  go/zodb: Prevent newly-created not-yet committed objects to go to ghost on deactivate
  go/zodb: Fix PActivate not to panic after an error
  go/zodb/fs1tools: Verify
  go/zodb/fs1tools: Dumper += DumpEndOK
  go/zodb: tests: Add a couple of missed t.Helper()
  go/zodb: tests: Teach tDB.Commit to return committed TID

When object is just created, it is not yet assigned an OID, but can be
reachable from other objects. The code that processes transaction can
reach to that new object and try to PActivate/PDeactivate it. And
currently PDeactivate will drop the object state completely.

Another example of object without an OID is Bucket embedded into a Tree
object. There, the code that scans the tree can reach to that bucket and
try to activate/deactivate it, leading, again, to dropping state of that bucket.

-> Fix it.

Persistent.PActivate used to panic when called the second time, if the
first time it hit an error. WCFS hit this in practice via object, that was
previously accessed and pinned into the cache, but later deleted in the
storage.

-> Fix PActivate to reset .loading on an error, so that next time PActivate is
called, it tries to trigger load again instead of panicking. Change doload
criteria from

	state==GHOST && refcnt==1

to

	state==GHOST && loading==nil

because now, after failed PActivate, refcnt can be != 0, if there are several
other PActivate calls that were waiting for the failed PActivate but did not
yet woke up.

Here is how added test fails without the fix:

    --- FAIL: TestActivateAfterDelete (1.65s)
    panic: t.zodb.MyObject(0000000000000065): activate: need to load, but .loading != nil [recovered]
            panic: t.zodb.MyObject(0000000000000065): activate: need to load, but .loading != nil

    goroutine 10085 [running]:
    testing.tRunner.func1.2(0x649020, 0xc000520660)
            /home/kirr/src/tools/go/go/src/testing/testing.go:1143 +0x332
    testing.tRunner.func1(0xc0001cb080)
            /home/kirr/src/tools/go/go/src/testing/testing.go:1146 +0x4b6
    panic(0x649020, 0xc000520660)
            /home/kirr/src/tools/go/go/src/runtime/panic.go:965 +0x1b9
    lab.nexedi.com/kirr/neo/go/zodb.(*Persistent).PActivate(0xc0001184d0, 0x6e8360, 0xc00019ac90, 0x0, 0x0)
            /home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/zodb/persistent.go:191 +0xce5
    lab.nexedi.com/kirr/neo/go/zodb.TestActivateAfterDelete(0xc0001cb080)
            /home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/zodb/persistent_test.go:786 +0x72c

Add utility to verify FileStorage data for consistency.
To verify we just need to iterate through all records, because
FileStorage driver performs all consistency checks by itself.

Mimic normal output to be the same as in fstest from ZODB/py.
Example runs of fstest.py and `fs1 verify` on a broken file:

    $ python ~/src/wendelin/z/ZODB/src/ZODB/scripts/fstest.py -v 1.fs
             4: transaction tid 0x03e044f6448c8022 #0
           213: transaction tid 0x03e044f646e044bb #1
    1.fs has data records that extend beyond the transaction record; end at 466

    $ fs1 verify -v 1.fs
             4: transaction tid 0x03e044f6448c8022 #0
           213: transaction tid 0x03e044f646e044bb #1
    2021/05/24 12:43:37 fsverify: 1.fs: 1.fs: transaction record @355: -> (iter data): 1.fs: data record @416: check: data record [..., 466) overlaps txn boundary [..., 458)

As can be seen, in fs1 case, the error contains more details: [start, end) of
both mismatching transaction and data records.

In addition to fstest-like verbosity, add progress-mode, where % of total
completion is printed in a style similar to one used by `fs1 verify-index`.

The Go-based implementation is also faster even when data is on HDD. For
example on a 73GB database provided by @jerome[1] fsrefs.py takes ~15 minutes
to run and occupy ~70-100% of CPU. On the other hand `fs1 verify` takes ~7
minutes to run and occupy ~ 20-30% of CPU.

Tests pending.

[1] nexedi/zodbtools!19 (comment 129480)

Some dumpers might want to print something at the end of their dump.
We will need this functionality for Verify (see next patch).