client: detect if monkey-patched code changes

neo/client/__init__.py

@@ -12,18 +12,25 @@
 #
 ##############################################################################
 
-# At the moment, no ZODB release include the following patches.
-# Later, this must be replaced by some detection mechanism.
 if 1:
-
+    from hashlib import md5
     from ZODB.Connection import Connection
 
+    def _check(f, *args):
+        h = md5(f.func_code.co_code).hexdigest()
+        assert h in args, h
+
     # Allow serial to be returned as late as tpc_finish
     #
     # This makes possible for storage to allocate serial inside tpc_finish,
     # removing the requirement to serialise second commit phase (tpc_vote
     # to tpc_finish/tpc_abort).
 
+    _check(Connection.tpc_finish,
+        'f50ed2e5a74f584fa1ecaf1cd70a6b15', # Python 2.6
+        'ab9b1b8d82c40e5fffa84f7bc4ea3a8b', # Python 2.7
+        )
+
     def tpc_finish(self, transaction):
         """Indicate confirmation that the transaction is done."""
 
@@ -39,6 +46,7 @@ if 1:
 #       to be able to read any updated data until we've had a chance
 #       to send an invalidation message to all of the other
 #       connections!
+        # <patch>
         serial = self._storage.tpc_finish(transaction, callback)
         if serial is not None:
             assert isinstance(serial, str), repr(serial)
@@ -49,6 +57,7 @@ if 1:
                     if obj is not None and obj._p_changed is not None:
                         obj._p_changed = 0
                         obj._p_serial = serial
+        # </patch>
         self._tpc_cleanup()
 
     Connection.tpc_finish = tpc_finish
@@ -62,6 +71,11 @@ if 1:
     # so this should not be done in afterCompletion(), and anyway, we don't
     # know any legitimate use of DB access outside a transaction.
 
+    _check(Connection.afterCompletion,
+        '70dfc3df8a455d9f663dec619c607eb5', # Python 2.6
+        'cd3a080b80fd957190ff3bb867149448', # Python 2.7
+        )
+
     def afterCompletion(self, *ignored):
         self._readCurrent.clear()
         # PATCH: do not call sync()

[Kirill Smelkov]

Hello @jm,

Recently with @Tyagov we were trying my wendelin.core tutorial[1] with NEO and when using it out of the box, i.e. pip install wendelin.core neoppod and then

root = dbopen('neo://dbname@master')

neo.client.Storage tries to monkey-patch Connection.tpc_finish with first verifying Connection.tpc_finish code is in expected state and "fails":

>>> import neo.client
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/kirr/tmp/trashme/1/local/lib/python2.7/site-packages/neo/client/__
    'ab9b1b8d82c40e5fffa84f7bc4ea3a8b', # Python 2.7
  File "/home/kirr/tmp/trashme/1/local/lib/python2.7/site-packages/neo/client/__
    assert h in args, h
AssertionError: b7b144f0a4865a272d4200e520193f1d

this happens e.g. with ZODB 4 in the following configuration:

(1) kirr@teco:~/tmp/trashme$ pip list
BTrees (4.3.1)
futures (3.0.5)
neoppod (1.6.3)
persistent (4.2.1)
pip (8.1.2)
pkg-resources (0.0.0)
setuptools (24.0.3)
six (1.10.0)
transaction (1.6.1)
trollius (2.1)
wheel (0.29.0)
zc.lockfile (1.2.1)
ZConfig (3.1.0)
zdaemon (4.1.0)
ZEO (5.0.0a0)
ZODB (4.4.2)
ZODB3 (3.11.0)
zodbpickle (0.6.0)
zope.interface (4.2.0)

The monkey-patches there in neo/client/__init__.py are for "Allow serial to be returned as late as tpc_finish" and for Connection.afterCompletion not to call sync()

I see our "Allow serial to be returned as late as tpc_finish" is already upstreamed:

https://github.com/zopefoundation/ZODB/commit/74fba08a

and is included in in ZODB >= 4.4.0

I also see many our patches are already upstreamed too. So my question here is:

• can we make this monkey-patching conditional and applied only to ZODB3 ?
• can we expect NEO to reasonably work with ZODB4 (and maybe 5) this way, or there are other blockers?

I ask because wendelin.core supports both ZODB3 and ZODB4 but for NEO current state is it works only on ZODB3 out-of-the-box.

Thanks beforehand,
Kirill

[1] https://www.nexedi.com/wendelin-Core.Tutorial.2016

[Julien Muchembled]

See the 'next' branch: I have already started to work on this. I am also impatient to make ERP5 & NEO use more recent versions of ZODB (another blocker for this is erp5!112 (merged)).

I prefer to finish some development on ZODB5 (remove support of old commit protocol) before anything else.

software/neoppod/software-zodb4.cfg is pinned to version 4.3.1. Can't you do the same for a while ?

[Ivan Tyagov]

@kirr , I suggest we go the way of fixing tutorial for now so it becomes usable.

[Kirill Smelkov]

@jm thanks for feedback. I already saw the next branch and was just curious about current status. Btw it feels great that ZODB is being revived this days. If there were no ZODB3 / ZODB eggs split we could just limit neoppod to require ZODB <= 4.4.0dev, but with the split and current setuptools state such limiting is not possible.

Thus I've adjusted the tutorial to use ZODB 4.3.X

Good luck with ZODB5 and thanks again,
Kirill

[Kirill Smelkov]

/cc @rafael