1. 05 Feb, 2019 8 commits
    • Julien Muchembled's avatar
      4b7acaa7
    • Julien Muchembled's avatar
      CMFActivity: remove old skin if any · afaa9d19
      Julien Muchembled authored
      afaa9d19
    • Julien Muchembled's avatar
      CMFActivity: drop DTML completely and use consecutive uids when possible · d64887cb
      Julien Muchembled authored
      This moves the remaining DTML queries to Python, dropping the 'activity' skin.
      
      Dealing with conflicts of uids is easier if the inserted uids are consecutive:
      now, only 1 random value is generated, as base uid. This also preserves the
      order of insertion, which is wanted for performance reasons:
      - No more random write in the primary index.
      - When modifying several lines of several documents, 1 document being processed
        at a time, we'd like that any grouped activity (usually indexation) follows
        the same order, so that a processing node prefer many lines from a few
        documents instead of mixing lines from too many documents at the same time.
        This is usually better for caches.
      d64887cb
    • Julien Muchembled's avatar
      CMFActivity: speed up console activity watcher · bcaf44a4
      Julien Muchembled authored
      Average age of activities is dropped because it would become
      too complicated to implement and it's useless information.
      bcaf44a4
    • Julien Muchembled's avatar
      CMFActivity: remove processing/processing_date columns and improve watchers · 499d8f8b
      Julien Muchembled authored
      The original goal was to improve performance by removing the
      `processing_node_processing` index and the queries that modified
      these 2 useless columns.
      499d8f8b
    • Julien Muchembled's avatar
      CMFActivity: do not refresh view too often when checking messages dependencies · a32c8a42
      Julien Muchembled authored
      The root call to getExecutableMessageList (i.e. the one from distribute)
      is fast enough and won't hold old revisions of the database for too long.
      It is also completely read-only so it won't lock anything.
      
      This caused useless communication with the server.
      a32c8a42
    • Julien Muchembled's avatar
    • Julien Muchembled's avatar
      CMFActivity: validate a message with a single SQL request per queue · 7e387bcb
      Julien Muchembled authored
      As shown in the following example, on a big catalog table,
      MariaDB is able to use several indices at the same time
      ('...' are obfuscated unique values):
      
        > analyze select SQL_NO_CACHE uid, relative_url from catalog where reference='...' OR relative_url='...';
        +------+-------------+---------+-------------+------------------------+------------------------+---------+------+------+--------+----------+------------+--------------------------------------------------+
        | id   | select_type | table   | type        | possible_keys          | key                    | key_len | ref  | rows | r_rows | filtered | r_filtered | Extra                                            |
        +------+-------------+---------+-------------+------------------------+------------------------+---------+------+------+--------+----------+------------+--------------------------------------------------+
        |    1 | SIMPLE      | catalog | index_merge | Reference,relative_url | Reference,relative_url | 768,767 | NULL |    2 |   2.00 |   100.00 |     100.00 | Using union(Reference,relative_url); Using where |
        +------+-------------+---------+-------------+------------------------+------------------------+---------+------+------+--------+----------+------------+--------------------------------------------------+
        1 row in set (0.00 sec)
      
      So mixing different dependency types with OR should be fine
      (no need to split into more subqueries and join with UNION).
      7e387bcb
  2. 18 Jan, 2019 3 commits
  3. 13 Jan, 2019 1 commit
    • Julien Muchembled's avatar
      Fix reindexing with custom grouping using 'group_id' · 707058d7
      Julien Muchembled authored
      This fixes:
      
        Traceback (innermost last):
          Module Products.CMFActivity.ActivityTool, line 1373, in invokeGroup
            traverse(method_id)(expanded_object_list)
          Module Products.ERP5Catalog.CatalogTool, line 946, in catalogObjectList
            super(CatalogTool, self).catalogObjectList(tmp_object_list, **m.kw)
          Module Products.ZSQLCatalog.ZSQLCatalog, line 813, in catalogObjectList
            **kw
        TypeError: catalogObjectList() got an unexpected keyword argument 'group_id'
      707058d7
  4. 08 Jan, 2019 3 commits
  5. 07 Jan, 2019 2 commits
    • Sebastien Robin's avatar
    • Julien Muchembled's avatar
      CMFActivity: better date ordering by using micro-precision · b82f3ba1
      Julien Muchembled authored
      Originally, uids somehow sorted messages by date of insertion, in particular
      for those that were created within the same second. But since random uids,
      such messages became validated or processed in random order.
      
      Note however that by default, messages created in the same transaction all have
      exactly the same date, so commit a42da4de
      ("CMFActivity: Do not use offset for scanning messages to validate.")
      forces us to keep the ordering on uids (in addition to priority/date).
      
      Existing instances will upgrade automatically, using the already existing code
      to upgrade tables in a generic way. You should see the following logs:
      
          INFO CMFActivity 'message_queue' table upgraded
          ALTER TABLE message_queue
            MODIFY COLUMN date datetime(6) NOT NULL AFTER uid,
            MODIFY COLUMN processing_date datetime(6) DEFAULT NULL AFTER processing
          INFO CMFActivity 'message_job' table upgraded
          ALTER TABLE message_job
            MODIFY COLUMN date datetime(6) NOT NULL AFTER uid,
            MODIFY COLUMN processing_date datetime(6) DEFAULT NULL AFTER processing
          INFO CMFActivity 'message' table upgraded
          ALTER TABLE message
            MODIFY COLUMN date datetime(6) NOT NULL AFTER uid,
            MODIFY COLUMN processing_date datetime(6) DEFAULT NULL AFTER processing
      
      
      /reviewed-on nexedi/erp5!820
      b82f3ba1
  6. 02 Jan, 2019 1 commit
  7. 31 Dec, 2018 1 commit
  8. 21 Dec, 2018 2 commits
  9. 20 Dec, 2018 2 commits
  10. 17 Dec, 2018 4 commits
  11. 14 Dec, 2018 2 commits
  12. 13 Dec, 2018 1 commit
    • Arnaud Fontaine's avatar
      Revert "mark file uploading tests as expected failure." as these Functional... · dd528bae
      Arnaud Fontaine authored
      Revert "mark file uploading tests as expected failure." as these Functional Tests are not supposed to fail.
      
      This reverts commit ade16831.
      
      enablePrivilege, and thus UniversalFileRead used to upload files, was disabled
      in Firefox 17. Since SlapOS Firefox has been upgraded, "The operation is insecure"
      error is raised when uploading files. However, this doesn't mean that these tests
      should be expected to fail (and if they are, they should be removed instead).
      dd528bae
  13. 12 Dec, 2018 1 commit
    • Jérome Perrin's avatar
      accounting: restrict restarting accounting period to Assignor · 366a760e
      Jérome Perrin authored
      This workflow involved an Assignee who can open, close, re-open and an
      Assignor who can close definitively.
      This is usually configured so that accountants are Assignee and CFO is
      Assignor.
      
      We realized that re-opening a Period that was previously closed is
      something we don't want the accountants to do without CFO's approval.
      
      To support this configuration, we only allow Assignor to re-open.
      
      Now Assignee can open and close temporarily and Assignor can re-open and
      close definitively.
      
      /reviewed-on !813
      366a760e
  14. 11 Dec, 2018 4 commits
  15. 10 Dec, 2018 5 commits