bigfile/zodb: Resync _ZBigFileH to Connection.transaction_manager on every connection reopen
This continues c7c01ce4 (bigfile/zodb: ZODB.Connection can migrate between threads on close/open and we have to care): Until now we were retrieving zconn.transaction_manager on _ZBigFileH init, and further using that transaction manager for every connection reopen. However that is not correct because on every reopen connection can be given new transaction manager. We were not practically hitting the bug because until recently ZODB was, by default, using the same ThreadTransactionManager manager instance as Connection.transaction_manager for all connections, and not doing all steps needed to keep _ZBigFileH.transaction_manager in sync to Connection was forgiven - a particular transaction manager that was used was TransactionManager instance implicitly associated with current thread by global threading.Local transaction.manager . However starting from ZODB 5.5.0 Connection code was changed to remember as .transaction_manager the particular TransactionManager instance without any threading.Local games: https://github.com/zopefoundation/ZODB/commit/b6ac40f153 https://github.com/zopefoundation/ZODB/issues/208 https://github.com/zopefoundation/ZODB/pull/226 Given that we were not syncing properly that broke wendelin.core tests, for example: bigfile/tests/test_filezodb.py::test_bigfile_filezodb_vs_conn_migration Exception in thread Thread-1: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner self.run() File "/usr/lib/python2.7/threading.py", line 754, in run self.__target(*self.__args, **self.__kwargs) File "/home/kirr/src/wendelin/wendelin.core/bigfile/tests/test_filezodb.py", line 401, in T11 transaction.commit() # should be nothing File "/home/kirr/src/wendelin/venv/z-dev/local/lib/python2.7/site-packages/transaction/_manager.py", line 252, in commit return self.manager.commit() File "/home/kirr/src/wendelin/venv/z-dev/local/lib/python2.7/site-packages/transaction/_manager.py", line 131, in commit return self.get().commit() File "/home/kirr/src/wendelin/venv/z-dev/local/lib/python2.7/site-packages/transaction/_transaction.py", line 298, in commit self._synchronizers.map(lambda s: s.beforeCompletion(self)) File "/home/kirr/src/wendelin/venv/z-dev/local/lib/python2.7/site-packages/transaction/weakset.py", line 61, in map f(elt) File "/home/kirr/src/wendelin/venv/z-dev/local/lib/python2.7/site-packages/transaction/_transaction.py", line 298, in <lambda> self._synchronizers.map(lambda s: s.beforeCompletion(self)) File "/home/kirr/src/wendelin/wendelin.core/bigfile/file_zodb.py", line 671, in beforeCompletion assert txn is zconn.transaction_manager.get() AssertionError What is happening here is that one thread used the connection and ZBigFile/_ZBigFileH associated with it, then the connection was closed and released to DB pool. Then the connection was reopened but by another thread and thus with different TransactionManager instance and oops - _ZBigFileH.transaction_manager is different because it is TransactionManager instance that was used by the first thread. Fix it by resyncing _ZBigFileH.transaction_manager on every connection reopen. No new test as existing tests already cover the problem when run with ZODB >= 5.5.0 .
Showing
Please register or sign in to comment