1. 04 Nov, 2021 25 commits
  2. 02 Nov, 2021 1 commit
  3. 27 Oct, 2021 1 commit
  4. 26 Oct, 2021 4 commits
  5. 25 Oct, 2021 1 commit
  6. 22 Oct, 2021 6 commits
    • Jérome Perrin's avatar
      core: Extend Category import with an option to force deletion of categories in use · 78cf1c94
      Jérome Perrin authored
      Category spreadsheet import did not delete categories if some documents
      where related to these categories. While this may make sense when using
      this import in site with already some data, in many cases, we want to
      be able to delete the categories, so that the result is that the category
      tree match the imported spreadsheet.
      
      This extend the dialog to support a new "force delete" mode that delete
      categories even if they are use, leaving the documents using categories
      with broken relations.
      78cf1c94
    • Vincent Pelletier's avatar
      ERP5Type/patches/WSGIPublisher: Avoid boolean evaluation of foreign object. · 679467c9
      Vincent Pelletier authored
      Spotted when comparing this code with ZPublisher.Publish.
      679467c9
    • Vincent Pelletier's avatar
      ERP5Type/patches/WSGIPublisher: Emulate HTTPResponse.write better. · 68527217
      Vincent Pelletier authored
      Once a response has been written to, the status and headers are supposed
      to have been sent over the network. So any future change to them is moot.
      Likewise, setBody does nothing: only write can continue appending to the
      existing body.
      As in the case of WSGI the response does not actually get sent just yet:
      - lock the body
      - store finalize() result
      Also, make the order in which body and stdout are returned consistent with
      HTTPResponse: body is returned first (as part of HTTPResponse.write's call
      to self.outputBody, which calls self.__str__, which emits body), and then
      stdout content.
      Also, add support for cases whre the existing body is a file or an
      IUnboundStreamIterator.
      68527217
    • Vincent Pelletier's avatar
      ERP5Type/patches/WSGIPublisher: Consider the error handled if err_hook reraised. · 7a6e04c1
      Vincent Pelletier authored
      err_hook may re-raise when it is specifically told not to render an error
      page. Such exception does not mean err_hook failed, so it should not be
      re-raised by WSGIPublisher: re-raising in WSGIPublisher means handing the
      exception over to waitress, which will have no other choice than producing a
      generic error page that we have no control over.
      7a6e04c1
    • Vincent Pelletier's avatar
      ERP5Type/patches/WSGIPublisher: Ensure transaction abort. · abf2ac13
      Vincent Pelletier authored
      Transaction must be aborted even if the state is getting so bad that the
      exception is being re-raised. Leaving an open-ended transaction is never
      acceptable.
      abf2ac13
    • Vincent Pelletier's avatar
      CMFActivity.Activity.SQLBase: Reduce the number of deadlocks · 18b5e4ed
      Vincent Pelletier authored
      MariaDB seems to be using inconsistent lock acquisition order when
      executing the activity reservation queries. As a consequence, it produces
      internal deadlocks, which it detects. Upon detection, it kills one of the
      involved query, which causes message reservation to fail, despite the
      presence of executable activities.
      To avoid depending on MariaDB internal lock acquisition order, acquire an
      explicit table-scoped lock before running the activity reservation queries.
      
      On an otherwise-idle 31 processing node cluster with the following
      activities spawned, designed to stress activity reservation queries
      (many ultra-short activities being executed one at a time):
        active_getTitle = context.getPortalObject().portal_catalog.activate(
          activity='SQLQueue',
          priority=5,
          tag='foo',
        ).getTitle
        for _ in xrange(40000):
          active_getTitle()
      the results are:
      - a 26% shorter activity execution time: from 206s with the original code
        to 152s
      - a 100% reduction in reported deadlocks from 300 with the original code
        to 0
      
      There is room for further improvements at a later time:
      - tweaking the amount of time spent waiting for this new lock to be
        available, set for now at 1s.
      - possibly bypassing this lock altogether when there are too few processing
        nodes simultaneously enabled, or even in an adaptive reaction to
        deadlock errors actually happening.
      - cover more write accesses to these tables with the same lock
      
      From a production environment, it appears that the getReservedMessageList
      method alone is involved in 95% of these deadlocks, so for now this change
      only targets this method.
      18b5e4ed
  7. 18 Oct, 2021 2 commits