Simplify MVCC by determining transaction start time using lastTransaction.
This implements: https://github.com/zopefoundation/ZODB/issues/50 Rather than watching invalidations, simply use 1 + the storages lastTransaction, which is equivalent to but much simpler than waiting for the first invalidation after a transaction starts. More importantly, it means we can always use loadBefore and get away from load. We no longer have to worry about ordering of invalidations and load() results. Much thanks to NEO for pointing the way toward this simplification! Implementing this initially caused a deadlock, because DB.open() called Connection.open() while holding a database lock and Connection.open() now calls IStotage.lastTransaction(), which acquires a storage lock. (It's not clear that lastTransaction() really needs a storage lock.) Meanwhile, IStotage.tpc_finish() calls a DB function that requires the DB lock while holding the storage lock. Fixing this required moving the call to Connection.open() outside the region where the DB lock was held. To debug the problem above, I greatly improved lock-debugging support. Now all of the ZODB code imports Lock, RLock and Condition from ZODB.utils. If the DEBUG_LOCKING is set to a non-empty value, then these are wrapped in such a way that debugging information is printed as they are used. This made spotting the nature of the deadlock easier. Of course, a change this basic broke lots of tests. Most of the breakage arises from the fact that connections now call lastTransaction on storages at transaction boundaries. Lots of tests didn't clean up databases and connections properly. I fixed many tests, but ultimately gave up and added some extra cleanup code that violated transaction-manager underware (and the underware's privates) to clear transaction synchonizers in test setup and tear-down. I plan to add a transaction manager API for this and to use it in a subsequent PR. This tests makes database and connection hygiene a bit more important, especially for tests, because a connection will continue to interact with storages if it isn't properly closed, which can lead to errors if the storage is closed. I chose not to swallow these errors in Connection, choosing rather to clean up tests. The thread debugging and test changes make this PR larger than I would have liked. Apologies in advance to the reviewers.
Showing with 349 additions and 290 deletions