Currently loadBefore and prefetch spawn async protocol.load_before task,
and, after waking up on its completion, populate the cache with received
data. But in between the time when protocol.load_before handler is run
and the time when protocol.load_before caller wakes up, there is a
window in which event loop might be running some other code, including
the code that handles invalidateTransaction messages from the server.

This means that cache updates and cache invalidations can be processed on
the client not in the order that server sent it. And such difference in
the order can lead to data corruption if e.g server sent

	<- loadBefore oid serial=tid1 next_serial=None
	<- invalidateTransaction tid2 oid

and client processed it as

	invalidateTransaction tid2 oid
	cache.store(oid, tid1, next_serial=None)

because here the end effect is that invalidation for oid@tid2 is not
applied to the cache.

The fix is simple: perform cache updates right after loadBefore reply is
received.

Fixes: https://github.com/zopefoundation/ZEO/issues/155

The fix is based on analysis and initial patch by @jmuchemb:

https://github.com/zopefoundation/ZEO/issues/155#issuecomment-581046248

A tests corresponding to the fix is coming coming through

https://github.com/zopefoundation/ZEO/pull/170  and
https://github.com/zopefoundation/ZODB/pull/345

For the reference, my original ZEO-specific data corruption reproducer
is here:

https://github.com/zopefoundation/ZEO/issues/155#issuecomment-577602842
https://lab.nexedi.com/kirr/wendelin.core/blob/ecd0e7f0/zloadrace5.py

/cc @jamadden, @dataflake, @jimfulton
/reviewed-by @jmuchemb, @d-maurer
/reviewed-on https://github.com/zopefoundation/ZEO/pull/169

tests: Factor "Open another ClientStorage to the same server" into generic _new_storage_client() (#170)

This allows ZODB tests to recognize ZEO as client-server storage and so
make "load vs external invalidate" test added in
https://github.com/zopefoundation/ZODB/pull/345 to reproduce data
corruption problem reported at
https://github.com/zopefoundation/ZEO/issues/155.

For the reference: that problem should be fixed by
https://github.com/zopefoundation/ZEO/pull/169.

We drop

    # It's hard to find the actual address.
    # The rpc mgr addr attribute is a list.  Each element in the
    # list is a socket domain (AF_INET, AF_UNIX, etc.) and an
    # address.

note because at the time it was added (81f586c4) it came with

    addr = self._storage._rpc_mgr.addr[0][1]

but nowdays after 03867188 getting to server address is just by ClientStorage._addr.

/reviewed-on https://github.com/zopefoundation/ZEO/pull/170
/reviewed-by @d-maurer

kirr:

this fixes "uvloop"-flavour CI failures where things were failing to
install with

    RuntimeError: uvloop requires Python 3.7 or greater
    An error occurred when trying to install /tmp/tmph6b7orkdget_dist/uvloop-0.15.2.tar.gz. Look above this message for any errors that were output by easy_install.
    While:
      Installing test.
      Getting distribution for 'uvloop>=0.5.1'.

(see e.g. https://travis-ci.com/github/zopefoundation/ZEO/jobs/498522609)

Starting from 1999 (b3805a2f "just getting started") only modified - not
just created - objects were included into ZEO invalidation messages:

https://github.com/zopefoundation/ZEO/commit/b3805a2f#diff-52fb76aaf08a1643cdb8fdaf69e37802R126-R127

In 2000 this behaviour was further changed to not send invalidation
message at all if the only objects a transaction has were the created ones:

https://github.com/zopefoundation/ZEO/commit/230ffbe8#diff-52fb76aaf08a1643cdb8fdaf69e37802L163-R163

In 2016 the latter was reconsidered as bug and fixed in ZEO5 because
ZODB5 relies more heavily on MVCC semantic and needs to be notified
about every transaction committed to storage to be able to properly
update ZODB.Connection view:

https://github.com/zopefoundation/ZEO/commit/02943acd#diff-52fb76aaf08a1643cdb8fdaf69e37802L889-R834
https://github.com/zopefoundation/ZEO/commit/9613f09b

In 2020, with this patch, I'm proposing to reconsider initial "send only
modified, not created objects" as bug, and include both modified and
just created objects into invalidation messages at least for the
following reasons:

- a ZODB client (not necessarily native ZODB/py client) can maintain
  raw cache for the storage. If such client tries to load an oid at
  database view when that object did not existed yet, gets "no object"
  reply and stores that information into raw cache, to properly invalidate
  the cache it needs an invalidation message from ZODB server that
  _includes_ created object.

- tools like `zodb watch` [1,2,3] don't work properly (give incorrect output)
  if not all objects modified/created by a transaction are included into
  invalidation messages.

- similarly to `zodb watch`, a monitoring tool, that would want to be
  notified of all created/modified objects, won't see full
  database-change picture, and so won't work properly without knowing
  which objects were created.

- wendelin.core 2 - which builds data from ZODB BTrees and data objects
  into virtual filesystem - needs to get invalidation messages with both
  modified and created objects to properly implement its own lazy
  invalidation and isolation protocol for file blocks in OS cache: when
  a block of file is accessed, all clients, that have this block mmaped,
  need to be notified and asked to remmap that block into particular
  revision of the file depending on a client's view of the filesystem and
  database [4,5].

  To compute to where a client needs to remmap the block, WCFS server
  (that in turn acts as ZODB client wrt ZEO/NEO server), needs to be able
  to see whether client's view of the filesystem is before object creation
  (and then ask that client to pin that block to hole), or after creation
  (and then ask the client to pin that block to corresponding revision).

  This computation needs ZODB server to send invalidation messages in
  full: with both modified and just created objects.

The patch is simple - it removes `if serial != b"\0\0\0\0\0\0\0\0"`
before queuing oid into ZEOStorage.invalidated, and adjusts the tests
correspondingly. From my point of view and experience, in practice, this
patch should not cause any compatibility break nor performance regressions.

Thanks beforehand,
Kirill

/cc @jimfulton

[1] https://lab.nexedi.com/kirr/neo/blob/ea53a795/go/zodb/zodbtools/watch.go
[2] kirr/neo@e0d59f5d
[3] kirr/neo@c41c2907

[4] https://lab.nexedi.com/kirr/wendelin.core/blob/1efb5876/wcfs/wcfs.go#L94-182
[5] https://lab.nexedi.com/kirr/wendelin.core/blob/1efb5876/wcfs/client/wcfs.h#L20-71

Fix tests with ZODB >= 5.6.0.

See https://github.com/zopefoundation/ZODB/pull/207

Update package link from pypi.python.org to pypi.org

Documentation draft

Replace deprecated occurences of Thread.isAlive() by Thread.is_alive()

The variable is named `address` and is called `addr` in the following lines. This PR use `address` consistently.

Use of --configure option instead of --configuration

fix Py3 SSL tests by providing proper CA test certificates

Fix typo

Use a temporary directory in tests

Otherwise when you run `tox` this test will create a `data.fs` using
Python 2 and then the same test will fail on Python 3 because there
already is a `data.fs` in the current working directory, but it has an
incompatible format.