• Kirill Smelkov's avatar
    tests: Force-close ZODB connections in teardown, that testing code forgot to explicitly close · 5a5ed2c7
    Kirill Smelkov authored
    If a test forgets to explicitly close ZODB connection it was using, this
    connection stays alive in transaction synchronizers (it is a weakset),
    and continues to be used on e.g. transaction.commit() when all
    synchronizers are invoked. This could lead to crashes like below when
    underlying ZODB storage was closed by test module teardown and testing
    moved on to another test module:
    
        $ WENDELIN_CORE_TEST_DB="<neo>" py.test  bigfile/tests/test_filezodb.py::test_bigfile_zblk1_zdata_reuse lib/tests/test_zodb.py
        ======= test session starts ========
        platform linux2 -- Python 2.7.14+, pytest-3.5.0, py-1.5.3, pluggy-0.6.0
        rootdir: /home/kirr/src/wendelin/wendelin.core, inifile:
        collected 2 items
    
        bigfile/tests/test_filezodb.py .                                     [ 50%]
        lib/tests/test_zodb.py F                                             [100%]
    
        ______ test_deactivate_btree _______
    
            def test_deactivate_btree():
                root = dbopen()
                # init btree with many leaf nodes
                leafv = []
                root['btree'] = B = IOBTree()
                for i in range(10000):
                    B[i] = xi = XInt(i)
                    leafv.append(xi)
        >       transaction.commit()
    
        lib/tests/test_zodb.py:56:
        _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
        ../venv/z5/local/lib/python2.7/site-packages/transaction/_manager.py:131: in commit
            return self.get().commit()
        ../venv/z5/local/lib/python2.7/site-packages/transaction/_transaction.py:316: in commit
            self._synchronizers.map(lambda s: s.afterCompletion(self))
        ../venv/z5/local/lib/python2.7/site-packages/transaction/weakset.py:62: in map
            f(elt)
        ../venv/z5/local/lib/python2.7/site-packages/transaction/_transaction.py:316: in <lambda>
            self._synchronizers.map(lambda s: s.afterCompletion(self))
        ../venv/z5/local/lib/python2.7/site-packages/ZODB/Connection.py:757: in afterCompletion
            self.newTransaction(transaction, False)
        ../venv/z5/local/lib/python2.7/site-packages/ZODB/Connection.py:737: in newTransaction
            invalidated = self._storage.poll_invalidations()
        ../venv/z5/local/lib/python2.7/site-packages/ZODB/mvccadapter.py:131: in poll_invalidations
            self._start = p64(u64(self._storage.lastTransaction()) + 1)
        _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    
        self = <neo.client.Storage.Storage object at 0x7ffa1be8d410>
    
            def lastTransaction(self):
                # Used in ZODB unit tests
        >       return self.app.last_tid
        E       AttributeError: 'NoneType' object has no attribute 'last_tid'
    
        ../../neo/src/lab.nexedi.com/kirr/neo/neo/client/Storage.py:181: AttributeError
    
    where NEO's Storage.app is None because the storage was closed.
    
    ----
    
    To avoid such kind of failures make sure TestDB.teardown() always closes
    all ZODB connections that were ever opened via TestDB.dbopen().
    
    Add a warning about such force-closing with information about corresponding
    connection and code place that created it, so that it is easy to
    understand which test needs a fix.
    
    /suggested-by @jm
    5a5ed2c7
testing.py 9.83 KB