- 04 May, 2016 1 commit
-
-
Jim Fulton authored
This implements: https://github.com/zopefoundation/ZODB/issues/50 Rather than watching invalidations, simply use 1 + the storages lastTransaction, which is equivalent to but much simpler than waiting for the first invalidation after a transaction starts. More importantly, it means we can always use loadBefore and get away from load. We no longer have to worry about ordering of invalidations and load() results. Much thanks to NEO for pointing the way toward this simplification! Implementing this initially caused a deadlock, because DB.open() called Connection.open() while holding a database lock and Connection.open() now calls IStotage.lastTransaction(), which acquires a storage lock. (It's not clear that lastTransaction() really needs a storage lock.) Meanwhile, IStotage.tpc_finish() calls a DB function that requires the DB lock while holding the storage lock. Fixing this required moving the call to Connection.open() outside the region where the DB lock was held. To debug the problem above, I greatly improved lock-debugging support. Now all of the ZODB code imports Lock, RLock and Condition from ZODB.utils. If the DEBUG_LOCKING is set to a non-empty value, then these are wrapped in such a way that debugging information is printed as they are used. This made spotting the nature of the deadlock easier. Of course, a change this basic broke lots of tests. Most of the breakage arises from the fact that connections now call lastTransaction on storages at transaction boundaries. Lots of tests didn't clean up databases and connections properly. I fixed many tests, but ultimately gave up and added some extra cleanup code that violated transaction-manager underware (and the underware's privates) to clear transaction synchonizers in test setup and tear-down. I plan to add a transaction manager API for this and to use it in a subsequent PR. This tests makes database and connection hygiene a bit more important, especially for tests, because a connection will continue to interact with storages if it isn't properly closed, which can lead to errors if the storage is closed. I chose not to swallow these errors in Connection, choosing rather to clean up tests. The thread debugging and test changes make this PR larger than I would have liked. Apologies in advance to the reviewers.
-
- 29 Apr, 2016 2 commits
-
-
Tres Seaver authored
Incididentally, should make the Py26 winbot happy too, until we can get it disabled.
-
Tres Seaver authored
Fixed test failure introduced in pull-request 51
-
- 28 Apr, 2016 2 commits
-
-
Jim Fulton authored
Fix possible data corruption after FileStorage is truncated to roll back a transaction
-
https://github.com/zopefoundation/ZODB/pull/51Jim Fulton authored
The PR depended on system time being UTC.
-
- 27 Apr, 2016 4 commits
-
-
Julien Muchembled authored
-
Julien Muchembled authored
Multi-threaded IO support, which is new to ZODB 3.10, allows clients to read data (load & loadBefore) even after tpc_vote has started to write a new transaction to disk. This is done by using different 'file' objects. Issues start when a transaction is rolled back after data has been appended (using the writing file object). Truncating is not enough because the FilePool may have been used concurrently to read the end of the last transaction: file objects have their own read buffers which, in this case, may also contain the beginning of the aborted transaction. So a solution is to invalidate read buffers whenever they may contain wrong data. This patch does it on truncation, which happens rarely enough to not affect performance. We discovered this bug in the following conditions: - ZODB splitted in several FileStorage - many conflicts in the first committed DB, but always resolved - unresolved conflict in another DB If the transaction is replayed with success (no more conflict in the other DB), a subsequent load of the object that could be resolved in the first DB may, for example, return a wrong serial (tid of the aborted transaction) if the layout of the committed transaction matches that of the aborted one. The bug usually manifests with POSKeyError & CorruptedDataError exceptions in ZEO logs, for example while trying to resolve a conflict (and restarting the transaction does not help, causing Site Errors in Zope). But theorically, this could also cause silent corruption or unpickling errors at client side.
-
Julien Muchembled authored
-
Julien Muchembled authored
Cleanup dependencies, remove ZEO leftovers
-
- 25 Apr, 2016 2 commits
-
-
Julien Muchembled authored
-
Julien Muchembled authored
- remove now useless `zdaemon` dependency - remove ZEO documentation - update dependency list in README
-
- 04 Apr, 2016 1 commit
-
-
Jason Madden authored
patch time.time in historical_connections to avoid windows failures; fix race condition in pool.availableGC * patch time.time in historical_connections to avoid windows failures. * Also use a mock version of datetime.datetime.utcnow. We can't patch the method in-place because it's an extension type. * synchronize sleep with time * increment by less than the pool timeout to avoid a race condition. * Avoid mutating the dict as we iter over it. Seen in Python 3.4 tests: File "/home/travis/build/zopefoundation/ZODB/src/ZODB/historical_connections.txt", line 263, in historical_connections.txt Failed example: conn = db.open() Exception raised: Traceback (most recent call last): File "/opt/python/3.4.2/lib/python3.4/doctest.py", line 1324, in __run compileflags, 1), test.globs) File "<doctest historical_connections.txt[0]>", line 1, in <module> conn = db.open() File "/home/travis/build/zopefoundation/ZODB/src/ZODB/DB.py", line 761, in open self.historical_pool.availableGC() File "/home/travis/build/zopefoundation/ZODB/src/ZODB/DB.py", line 277, in availableGC for key, pool in self.pools.items(): RuntimeError: dictionary changed size during iteration
-
- 18 Mar, 2016 1 commit
-
-
Tres Seaver authored
Do3cc fix for45 master
-
- 17 Mar, 2016 6 commits
-
-
Patrick Gerken authored
-
Patrick Gerken authored
-
Patrick Gerken authored
-
Patrick Gerken authored
Nested connections try to clean up themselves multiple times. This does not work
-
Patrick Gerken authored
Drop support for Python 2.6 / 3.2.
-
Tres Seaver authored
3.2 is no longer supported by our packaging / CI tools. 2.6 is long out-of-support from Python devs.
-
- 16 Nov, 2015 1 commit
-
-
Jason Madden authored
Cleanup tox.ini and .travis.yml as mentioned in #42 (and Python 3.5)
-
- 13 Nov, 2015 1 commit
-
-
Jason Madden authored
-
- 07 Nov, 2015 4 commits
-
-
Jason Madden authored
- (Both) Remove repetition of deps. Only install tests_require deps explicitly. The previous comment in tox.ini about eggs in the working directory doesn't apply, per @mgedmin. - Setup Travis to use the local wheel cache for faster install times.
-
Marius Gedminas authored
-
-
Marius Gedminas authored
Make zodbpickle non-optional.
-
- 03 Nov, 2015 1 commit
-
-
Jason Madden authored
-
- 14 Jul, 2015 1 commit
-
-
Marius Gedminas authored
(IIRC a bug in python-modernize causes duplicate future imports)
-
- 09 Jun, 2015 1 commit
-
-
Marius Gedminas authored
-
- 02 Jun, 2015 4 commits
-
-
Tres Seaver authored
-
Tres Seaver authored
-
Tres Seaver authored
-
Marius Gedminas authored
Use PEP-426 environment markers
-
- 25 May, 2015 4 commits
-
-
Marius Gedminas authored
-
Marius Gedminas authored
Any attempts to use platform_python_implementation or implementation_name result in an "Invalid environment marker" error. Any attempts to use >= to compare python_version result in an "Invalid environment marker" error.
-
Marius Gedminas authored
Augh documentation for all of this is painfully suboptimal.
-
Marius Gedminas authored
Should fix #36
-
- 22 May, 2015 4 commits
-
-
Tres Seaver authored
-
Tres Seaver authored
-
Tres Seaver authored
-
Tres Seaver authored
-