1. 25 Oct, 2018 6 commits
    • Vincent Pelletier's avatar
      ERP5Catalog: Make user/group distinction scriptable. · d0260f5e
      Vincent Pelletier authored
      This allows breaking a circular dependency of catalog on itself: to
      index a local role, catalog must tell whether it's granted to a group or
      to a user, which used to happen using a catalog lookup (via PAS API),
      preventing catalog from being rebuilt from scratch.
      
      With this change, ERP5Site_filterUserIdSet must be implemented when
      custom role columns are used (viewable_owner is not concerned by this
      change).
      If it is implemented in a way which does not rely on catalog,
      catalog become rebuildable from a lone ZODB, without double-indexation
      of users nor security_uid explosion.
      Nothing prevents relying on catalog (which mayu be desired for
      backward-compatibility purposes), but such script will not be provided
      by generic ERP5, as the impossibility of recreating catalog from ZODB
      is considered a bug, and introducing such script amounts to introducing
      this bug.
      
      Create ERP5Site_filterUserIdList in tests which add custom viewable_*
      columns.
      d0260f5e
    • Vincent Pelletier's avatar
      ERP5Catalog: Reduce reliance on user lookup. · 4c514da5
      Vincent Pelletier authored
      Use Owner role to help filling user_set: owners must be users, never
      groups.
      Only look for users among group_ids when they are candidate for indexation
      into a catalog_role column. This should very significantly reduce the
      number of user lookups, even bringing it to zero on instance with default
      catalog setting of catalog_role_set = set(["viewable_owner"]) .
      Also, only look for users if there are candidates for lookup.
      Also, rename local variable to better describe its content.
      Also, use set.__iadd__ instead of set.update (shorter).
      4c514da5
    • Vincent Pelletier's avatar
      ERP5Catalog: Assorted simplifications. · 9542da22
      Vincent Pelletier authored
      Test loop invariant outside of loop.
      Use clearer and more consistent variable names.
      Avoid iterating twice when once is enough, allowing removal of
      "local_role_dict" local and simplification of corresponding dict
      structure.
      Avoid creating new instances when the existing one can be mutated.
      Evaluate dict as boolean instead of constructing an empty dict to compare
      against.
      Avoid using %-operator just to concatenate strings (allows en-passant
      typechecking).
      Drop unused locals.
      Do not use a dict when a set would be sufficient.
      9542da22
    • Vincent Pelletier's avatar
      ERP5Security.ERP5GroupManager: Move security category mapping lookup out. · 748046d0
      Vincent Pelletier authored
      So that it can be reused outside of group manager without having to
      duplicate the backward-compatibility order.
      Also, simplify code.
      748046d0
    • Vincent Pelletier's avatar
      ERP5Security.ERP5GroupManager: Coding style. · ae6b3070
      Vincent Pelletier authored
      Move variable initialisations closer to where they are actually used.
      Do not initialise a variable which will always be initilised again before
      its first use.
      Reduce argument indentation.
      Drop unhelpful comments.
      User document may not be a Person instance, use a more neutral variable
      name.
      Avoid single-use locals.
      Take advantage of zLOG.LOG's "error=True" idiom to not have to pull
      exception information ourselves.
      ae6b3070
    • Vincent Pelletier's avatar
  2. 24 Oct, 2018 7 commits
  3. 23 Oct, 2018 1 commit
    • Vincent Pelletier's avatar
      CMFCategory: Do not use __getitem__ to access a subobject. · 97480b84
      Vincent Pelletier authored
      It is extremely slow for some reason on large BTreeFolders inside
      BTreeFolder2's has_key (even on warm caches). Using get instead is
      significantly faster.
      Found by chance, calling getAcquiredCategoryList on a random Person when
      person_module contains of the order of magnitude of a million documents.
      97480b84
  4. 22 Oct, 2018 6 commits
  5. 20 Oct, 2018 2 commits
  6. 19 Oct, 2018 1 commit
    • Jérome Perrin's avatar
      ProcessingNodeTestCase: respect isSubscribed state of activity tool · 70be8872
      Jérome Perrin authored
      This is needed when running livetests in a runUnitTest instance, in the
      following scenario:
       - `runUnitTest --save test_bt:testXXX`
       - `runUnitTest --save --load`
       - run the live test from the browser
      
      because livetest way of processing activities is to unsubscribe at the
      beginning of the test and subscribe at the end to have control of
      activities when running .tic, if ProcessingNodeTestCase also process the
      activities automatically, they interfere and test often fail with errors like:
      
      ```
      File "ERP5Type/tests/ProcessingNodeTestCase.py", line 249, in tic
        raise RuntimeError(error_message)
      RuntimeError: tic is looping forever. These messages are pending: [('/erp5/portal_components/test.erp5.testSupportRequest', 'immediateReindexObject', 1, 0), ('/erp5/support_request_module/2617', 'immediateReindexObject', -1, 0)]
      ```
      
      /reviewed-on nexedi/erp5!779
      70be8872
  7. 17 Oct, 2018 2 commits
    • Jérome Perrin's avatar
      ui_test: simplify assertion · f1a85400
      Jérome Perrin authored
      This step just checks that multi relation field listbox shows a list of
      related document and that clicking on any item of the list leads to the
      related document.
      The order of related documents in the list has always been unspecified,
      but the first one was always 1 until recently.
      Because this test does not really care which one of the two related
      document was clicked, just assert that it's a document from foo_module.
      
      /reviewed-on nexedi/erp5!774
      f1a85400
    • Jérome Perrin's avatar
      Zelenium: do not include (HTML) link in case of expected failure · a636336a
      Jérome Perrin authored
      89115b88 is great, but with "testFunctionalAnonymousSelection" the
      result HTML is more than 40Mo.
      
      /reviewed-on nexedi/erp5!772
      a636336a
  8. 16 Oct, 2018 8 commits
  9. 15 Oct, 2018 1 commit
    • Vincent Pelletier's avatar
      CMFActivity: Do not use offset for scanning messages to validate. · a42da4de
      Vincent Pelletier authored
      This was inefficient for two reasons:
      - any message we could validate during current iteration means a message we
        did not consider is now in the range we just scanned. And it will not be
        considered until validation node starts over and scan this same range
        again.
      - "LIMIT x,1000" pattern on >1000 messages causes a quick-growing number of
        extra rows scanned by the SQL database just to skip the "x" first rows:
        at 2000 rows present it must scan 1000 + 2000 = 3000 rows for a complete
        loop over all pending activities. At 3k rows it must scan 6k rows.
        At 4k, 10k.
        While this is an overestimation (some rows should be possible to
        validate, so these would be scanned once only), this overhead grows so
        large that this overestimation can become negligible.
      
      Instead, use a range condition consistent with query's "SORT ON", which is
      already efficiently materialised by an index: SQL database just has to
      dive into the existing index to start just above the last message from
      previous iteration, and resume scanning from there, solving both issues
      listed above.
      a42da4de
  10. 12 Oct, 2018 2 commits
    • Vincent Pelletier's avatar
      ERP5Type: Patch AccessControl.owner.Owned.getWrappedOwner . · ea56fe72
      Vincent Pelletier authored
      This method is called on every __getattr__ (guarded_getattr, actually) of
      every restricted python scripts. Which means each "." in the code triggers:
      - traversal to relevant acl_user folder
      - a user lookup
      just to get the same value throughout the execution of a script (as its
      owner does not change during execution).
      Also, in practice we have extremely few possible owners: very few users are
      allowed to edit code in an ERP5 instance, and if such instance is managed
      using the upgrader alarms, they will even further reduce this owner set to
      System Processes only.
      On a real-world web page rendering, this reduces the total number of
      traversal calls from 1500 to 1100, getting rid of the two hottest spots:
      /acl_users and /$site_id/acl_users .
      ea56fe72
    • Tristan Cavelier's avatar
  11. 11 Oct, 2018 4 commits