1. 09 Feb, 2022 1 commit
    • Jérome Perrin's avatar
      periodicity: fix infinite loop with impossible combinations of week and months · 805dd17c
      Jérome Perrin authored
      Some combinations of periodicity, for example repeat every first week
      of the year and every month February are impossible (because the first
      week of the year is always in January) and such configurations caused
      infinite loops or probably overflow if we wait long enough.
      
      The algorithm being to try the next day until all constraints are met,
      it is not guaranteed to terminate.
      
      To make sure the algorithm terminate, we rely on the fact that calendars
      repeat after some time, so if after a few years we did not find a
      matching combination, we can stop retrying.
      
      according to https://en.wikipedia.org/wiki/Determination_of_the_day_of_the_week
      
      > Each leap year repeats once every 28 years, and every common year
      > repeats once every 6 years and twice every 11 years.
      
      so trying for 28 years should be enough to see all combinations
      805dd17c
  2. 08 Feb, 2022 5 commits
    • Jérome Perrin's avatar
      testSelectionTool: open connection in worker thread · ea340e2d
      Jérome Perrin authored
      In ZODB 5, with commit b6ac40f1 (Uses an unwrapped transaction manager,
      2018-10-14) the transaction is bound to the thread opening the
      connection.
      The previous pattern of opening transaction in the main thread and
      passing the already-open connection to the working thread caused the
      working thread connection to be managed by the main thread connection
      and in ZODB 5 cause the test to block.
      
      Fix this by passing a connection factory method and opening
      connection in working thread.
      Also simplify closing of connection by using a closing context manager.
      ea340e2d
    • Jérome Perrin's avatar
      ERP5Type/XMLExportImport: use zodbpickle pickler for OrderedPickler · 00238dfe
      Jérome Perrin authored
      With upcoming ZODB 5, oids (used as persistent references in pickles)
      are no longer str as it use to be with ZODB 4, but instances of
      zodbpickle.binary, which with zodbpickle 1 are a subclass of str on
      python2.
      
      OrderedPickler was a subclass of pickle.Pickler, the pickler from standard
      library, but this pickler was not able to use a str subclass for persistent
      references, when pickles are loaded with noload method, persistent_load
      is called with `None` instead of the actual string subclass instance.
      This was problematic in the XMLExportImport handling of business templates,
      because ZODB.serialize.referencesf was unable to find persistent references.
      The error was:
      
          ZODB-5.6.0-py2.7.egg/ZODB/serialize.py", line 664, in referencesf
              assert isinstance(reference, list)
          AssertionError
      
      because the reference was None.
      
      zodbpickle 2 changed to make zodbpickle.binary implemented in C, which
      was failing earlier, because pickle.Pickle can not pickle these objects,
      failing in an error like this:
      
          lib/python2.7/copy_reg.py", line 70, in _reduce_ex
              raise TypeError, "can't pickle %s objects" % base.__name__
          TypeError: can't pickle binary objects
      
      This change also simplify our own implementation, by dropping jython
      support and calling save_dict on the super class instead of copying the
      implementation.
      
      Further references:
      
      - minimal script to reproduce the issues:
      
      ```python
      from __future__ import print_function
      import io
      import pickle
      
      import zodbpickle
      import zodbpickle.pickle
      import zodbpickle.fastpickle
      
      class ExternalObject(object):
        def __init__(self, oid):
          self.oid = oid
      
      def persistent_id(obj):
        if isinstance(obj, ExternalObject):
          return obj.oid
      
      def persistent_load(persid):
        print('persistent_load called with persid', repr(persid))
      
      o = ExternalObject(oid=zodbpickle.binary("binary persid"))
      
      for pickler_class in pickle.Pickler, zodbpickle.pickle.Pickler:
      
        f = io.BytesIO()
        p = pickler_class(f, 1)
        p.persistent_id = persistent_id
        p.dump(o)
      
        print('dump with pickler %s:\n  %r' % (pickler_class, f.getvalue()))
      
        # ZODB uses this unpickler
        up = zodbpickle.fastpickle.Unpickler(io.BytesIO(f.getvalue()))
        up.persistent_load = persistent_load
        up.noload()
      ```
      
      ```console
      $ python2 repro.py # with zodbpickle 1
      dump with pickler pickle.Pickler:
        'ccopy_reg\n_reconstructor\nq\x00(czodbpickle\nbinary\nq\x01c__builtin__\nstr\nq\x02U\rbinary persidq\x03tq\x04Rq\x05Q.'
      persistent_load called with persid None
      dump with pickler zodbpickle.pickle_2.Pickler:
        'U\rbinary persidq\x00Q.'
      persistent_load called with persid 'binary persid'
      ```
      
      ```console
      $ python2 repro.py # with zodbpickle 2
      Traceback (most recent call last):
        File "repro.py", line 45, in <module>
          p.dump(o)
        File ".../lib/python2.7/pickle.py", line 224, in dump
          self.save(obj)
        File ".../lib/python2.7/pickle.py", line 273, in save
          self.save_pers(pid)
        File ".../lib/python2.7/pickle.py", line 340, in save_pers
          self.save(pid)
        File ".../lib/python2.7/pickle.py", line 306, in save
          rv = reduce(self.proto)
        File ".../lib/python2.7/copy_reg.py", line 70, in _reduce_ex
          raise TypeError, "can't pickle %s objects" % base.__name__
      TypeError: can't pickle binary objects
      ```
      
      * ZODB change starting to use zodbpickle.binary instead of str:
      12ee41c4 (-ZODB now uses pickle protocol 3 for both Python 2 and Python 3., 2018-03-26)
      Since of 5.4.0 release
      
      * zodbpickle change starting to use C objects for zodbpickle.binary:
      bbef98c (Implement zodbpickle.binary in C for Py27., 2019-11-12)
      Since of 2.0.0 release
      00238dfe
    • Jérome Perrin's avatar
      ProcessingNodeTestCase.tic: increase delay to 30 minutes · eb77cb53
      Jérome Perrin authored
      Now that we fail immediately in case of failure, the deadline can be
      safely increased, because it only protects against kind of infinite loops.
      
      Increasing the delay should fix RuntimeError: tic is looping forever
      errors with only messages in status -1, that we sometimes saw on testnodes.
      eb77cb53
    • Jérome Perrin's avatar
      ProcessingNodeTestCase.tic: fail as soon as one message had failed · b59999ad
      Jérome Perrin authored
      Now that tic retries until the deadline is reached or all messages has
      failed, it can lead to situations where developer have to wait until the
      deadline, when a message failed but other messages (typically scheduled
      to run after the failed message) were still running.
      
      By stopping as soon as one message is failed, in this scenario the
      developer does not need to wait until the deadline.
      b59999ad
    • Jérome Perrin's avatar
      00cd2af8
  3. 07 Feb, 2022 1 commit
  4. 04 Feb, 2022 1 commit
  5. 03 Feb, 2022 4 commits
  6. 02 Feb, 2022 18 commits
  7. 01 Feb, 2022 1 commit
  8. 31 Jan, 2022 1 commit
    • Vincent Pelletier's avatar
      Products.ERP5Type.Core.Predicate: Obey isEmptyCriterionValid . · 57463f30
      Vincent Pelletier authored
      When EmptyCriterionValid property is true, this method is expected to
      return a query which does not match any document. This only happens when
      query_list is empty, but because of category membership checking,
      query_list is never empty: it at least contains two ComplexQueries which
      themselves may contain an empty query list, and which match all documents.
      Calling getCategoryParameterDict with an empty list is dubious, but
      changing its behaviour in ZSQLCatalog may affect more than just predicates,
      so change the behaviour in Predicate class directly by checking whether
      there is any category being matched to begin with.
      57463f30
  9. 28 Jan, 2022 1 commit
  10. 27 Jan, 2022 1 commit
  11. 26 Jan, 2022 4 commits
  12. 24 Jan, 2022 2 commits
    • Vincent Pelletier's avatar
    • Jérome Perrin's avatar
      pdm: support defining function on supply lines · 622b2b80
      Jérome Perrin authored
      We make it possible to define function the same way it's currently
      possible to define accounts. The idea is similar: to be able to use this
      information in accounting generation rules.
      
      The typical accounting generation configuration using this will use the
      function defined on the supply line if any is defined and with a fallback
      to the function defined on business process' trade model path.
      
      For now we don't introduce such configuration by default, but this may
      become part of  the configuration generated by configurator some day.
      622b2b80