1. 18 Feb, 2022 1 commit
  2. 17 Feb, 2022 7 commits
  3. 16 Feb, 2022 10 commits
  4. 15 Feb, 2022 3 commits
  5. 14 Feb, 2022 1 commit
    • Vincent Pelletier's avatar
      erp5_core_test: Make testWorkflowHistoryList.TestDedup stable. · 3566c13f
      Vincent Pelletier authored
      The precise number of entries in a bucket depend on an estimation of the
      size of a pickle. The pickled data contains DateTime objects, making an
      equality test brittle:
      - DateTime's timezones are stored as strings (ex: 'GMT') whose length
        depend on Zope's timezone, which is variable
      - DateTime's time is stored as a floating-point value represented as a
        string whose length depends on the number of units and decimals are
        necessary to represent its value, both being variable (one based on when
        the test was run, the other based on clock precision and exact test
        execution timing)
      Instead, restore the originally-considered-acceptable boundary (24) and
      verify that the generated value is greater or equal to it.
      If 24 is considered too small to be acceptable, then it is a decision
      independent from the present change.
      3566c13f
  6. 13 Feb, 2022 1 commit
  7. 10 Feb, 2022 4 commits
  8. 09 Feb, 2022 1 commit
  9. 08 Feb, 2022 5 commits
    • Jérome Perrin's avatar
      testSelectionTool: open connection in worker thread · ea340e2d
      Jérome Perrin authored
      In ZODB 5, with commit b6ac40f1 (Uses an unwrapped transaction manager,
      2018-10-14) the transaction is bound to the thread opening the
      connection.
      The previous pattern of opening transaction in the main thread and
      passing the already-open connection to the working thread caused the
      working thread connection to be managed by the main thread connection
      and in ZODB 5 cause the test to block.
      
      Fix this by passing a connection factory method and opening
      connection in working thread.
      Also simplify closing of connection by using a closing context manager.
      ea340e2d
    • Jérome Perrin's avatar
      ERP5Type/XMLExportImport: use zodbpickle pickler for OrderedPickler · 00238dfe
      Jérome Perrin authored
      With upcoming ZODB 5, oids (used as persistent references in pickles)
      are no longer str as it use to be with ZODB 4, but instances of
      zodbpickle.binary, which with zodbpickle 1 are a subclass of str on
      python2.
      
      OrderedPickler was a subclass of pickle.Pickler, the pickler from standard
      library, but this pickler was not able to use a str subclass for persistent
      references, when pickles are loaded with noload method, persistent_load
      is called with `None` instead of the actual string subclass instance.
      This was problematic in the XMLExportImport handling of business templates,
      because ZODB.serialize.referencesf was unable to find persistent references.
      The error was:
      
          ZODB-5.6.0-py2.7.egg/ZODB/serialize.py", line 664, in referencesf
              assert isinstance(reference, list)
          AssertionError
      
      because the reference was None.
      
      zodbpickle 2 changed to make zodbpickle.binary implemented in C, which
      was failing earlier, because pickle.Pickle can not pickle these objects,
      failing in an error like this:
      
          lib/python2.7/copy_reg.py", line 70, in _reduce_ex
              raise TypeError, "can't pickle %s objects" % base.__name__
          TypeError: can't pickle binary objects
      
      This change also simplify our own implementation, by dropping jython
      support and calling save_dict on the super class instead of copying the
      implementation.
      
      Further references:
      
      - minimal script to reproduce the issues:
      
      ```python
      from __future__ import print_function
      import io
      import pickle
      
      import zodbpickle
      import zodbpickle.pickle
      import zodbpickle.fastpickle
      
      class ExternalObject(object):
        def __init__(self, oid):
          self.oid = oid
      
      def persistent_id(obj):
        if isinstance(obj, ExternalObject):
          return obj.oid
      
      def persistent_load(persid):
        print('persistent_load called with persid', repr(persid))
      
      o = ExternalObject(oid=zodbpickle.binary("binary persid"))
      
      for pickler_class in pickle.Pickler, zodbpickle.pickle.Pickler:
      
        f = io.BytesIO()
        p = pickler_class(f, 1)
        p.persistent_id = persistent_id
        p.dump(o)
      
        print('dump with pickler %s:\n  %r' % (pickler_class, f.getvalue()))
      
        # ZODB uses this unpickler
        up = zodbpickle.fastpickle.Unpickler(io.BytesIO(f.getvalue()))
        up.persistent_load = persistent_load
        up.noload()
      ```
      
      ```console
      $ python2 repro.py # with zodbpickle 1
      dump with pickler pickle.Pickler:
        'ccopy_reg\n_reconstructor\nq\x00(czodbpickle\nbinary\nq\x01c__builtin__\nstr\nq\x02U\rbinary persidq\x03tq\x04Rq\x05Q.'
      persistent_load called with persid None
      dump with pickler zodbpickle.pickle_2.Pickler:
        'U\rbinary persidq\x00Q.'
      persistent_load called with persid 'binary persid'
      ```
      
      ```console
      $ python2 repro.py # with zodbpickle 2
      Traceback (most recent call last):
        File "repro.py", line 45, in <module>
          p.dump(o)
        File ".../lib/python2.7/pickle.py", line 224, in dump
          self.save(obj)
        File ".../lib/python2.7/pickle.py", line 273, in save
          self.save_pers(pid)
        File ".../lib/python2.7/pickle.py", line 340, in save_pers
          self.save(pid)
        File ".../lib/python2.7/pickle.py", line 306, in save
          rv = reduce(self.proto)
        File ".../lib/python2.7/copy_reg.py", line 70, in _reduce_ex
          raise TypeError, "can't pickle %s objects" % base.__name__
      TypeError: can't pickle binary objects
      ```
      
      * ZODB change starting to use zodbpickle.binary instead of str:
      12ee41c4 (-ZODB now uses pickle protocol 3 for both Python 2 and Python 3., 2018-03-26)
      Since of 5.4.0 release
      
      * zodbpickle change starting to use C objects for zodbpickle.binary:
      bbef98c (Implement zodbpickle.binary in C for Py27., 2019-11-12)
      Since of 2.0.0 release
      00238dfe
    • Jérome Perrin's avatar
      ProcessingNodeTestCase.tic: increase delay to 30 minutes · eb77cb53
      Jérome Perrin authored
      Now that we fail immediately in case of failure, the deadline can be
      safely increased, because it only protects against kind of infinite loops.
      
      Increasing the delay should fix RuntimeError: tic is looping forever
      errors with only messages in status -1, that we sometimes saw on testnodes.
      eb77cb53
    • Jérome Perrin's avatar
      ProcessingNodeTestCase.tic: fail as soon as one message had failed · b59999ad
      Jérome Perrin authored
      Now that tic retries until the deadline is reached or all messages has
      failed, it can lead to situations where developer have to wait until the
      deadline, when a message failed but other messages (typically scheduled
      to run after the failed message) were still running.
      
      By stopping as soon as one message is failed, in this scenario the
      developer does not need to wait until the deadline.
      b59999ad
    • Jérome Perrin's avatar
      00cd2af8
  10. 07 Feb, 2022 1 commit
  11. 04 Feb, 2022 1 commit
  12. 03 Feb, 2022 4 commits
  13. 02 Feb, 2022 1 commit