client: replace global load lock by a per-oid one
This merge request contains merge conflicts
To merge this request, resolve these conflicts or ask someone with write access to this repository to merge it locally.
With ZODB5, Storage.load is almost not used anymore and
if not (tid or before_tid) and self.last_tid: before_tid = p64(u64(self.last_tid) + 1)
would be cheap enough to move it before the first access to the cache (a change suggested by @vpelletier). Then, maybe we could simplify thing, by reverting a2e278d5 somewhat (i.e. keep a single tid instead of a list).
However, I'd rather first try again to get rid of the next_tid for loads (suggested by @kirr).
@jm, thanks for heads up and for working on removing global load lock.
However, I'd rather first try again to get rid of the next_tid for loads (suggested by @kirr)
For the reference here is the data structure that is needed for cache to not require
I did not taught raw cache on zodb/go side to handle invalidations yet, but DB is already handling live cache invalidations - because wendelin.core needs it - and it also uses ΔTail:
before_tid = p64(u64(self.last_tid) + 1)- to me it is indeed step into right direction. Actually
beforeassociated with connection should be established when connection is opened and there it should either 1) get
storage.last_tidas locally cached without syncing to remote server, or 2) perform full round-trip to server to retrive last_tid from there. For correctness "2" is prefferred, however it is more expensive than "1". To me it should be thus an option for when new DB Connection is opened whether we should skip synking. Please see kirr/neo@151d8b79 which discusses this topic.
As we are talking about invalidations maybe this utility might be also useful sometimes for debug/monitor: kirr/neo@e0d59f5d.
With ZODB5, Storage.load is almost not used anymore
But this change is not
app-level, where the
loadmethod and the cache we are protecting access to by locks is shared amongst all load types ("loadEx", "loadSerial", "loadBefore", and plain old "load"), no ?
This is why this global lock is so bad to me and why I reported it now over 3 years ago: it prevents all loads of any kind by preventing even cache hits - maybe requested data is present in the cache and we make the thread wait on some totally unrelated load request which may be waiting on a large object to come through the network.