1. 21 Aug, 2019 1 commit
    • Julien Muchembled's avatar
      Publish Folder.recursiveReindexObject · fa64d07f
      Julien Muchembled authored
      With the removal of Folder_reindexAll, it has become laborious to reindex
      a whole module, and it was often done in an unsafe way (create a dummy wrapper
      script in custom skin folder, without thinking about security).
      fa64d07f
  2. 25 Jul, 2019 1 commit
  3. 20 Jun, 2019 1 commit
  4. 24 May, 2019 1 commit
  5. 21 May, 2019 2 commits
    • Vincent Pelletier's avatar
      ERP5Type.Core.Folder: Split _count on large containers. · 5dfae179
      Vincent Pelletier authored
      Divide conflict hot-spot to improve write performance: now, conflict risk
      will not be proportional to the number of zope processes, but only to the
      number of threads within considered process (...because there is no
      stable thread name, if there is one day and conflict splitting is needed,
      it will be easy to implement: just concatenate that name to node name in
      FragmentedLength._map).
      Automatically migrate to FragmentedCount on containers larger than 1k
      objects (this threshold may evolve).
      5dfae179
    • Vincent Pelletier's avatar
      ERP5.Tool.CategoryTool, Folder: Resolve multi-inheritance from CopyContainer · c5cd1820
      Vincent Pelletier authored
      These two classes inherits from two classes implementing CopyContainer API:
      - ours, via low-priority bases
      - another one, via higher-priority bases, because these classes need
        some non-ERP5 methods to override ERP5 methods.
      So after defining these classes, resolve inheritance priority by binding
      all CopyContainer methods provided by their ERP5 base class, overriding
      the ones provided by their non-ERP5 bases while still allowing their ERP5
      base class to override the CopyContainer method.
      c5cd1820
  6. 14 Mar, 2019 2 commits
  7. 14 Dec, 2018 1 commit
  8. 02 Oct, 2018 1 commit
  9. 14 Sep, 2018 2 commits
  10. 24 Aug, 2018 1 commit
    • Julien Muchembled's avatar
      fixup! Folder: Make recursiveReindexObject scalable by calling _recurseCallMethod · 614a8349
      Julien Muchembled authored
      With commit babbf56f, some objects that were
      not indexed are now indexed, and this fails when these objects can't be indexed.
      
      An exemple is when a PythonScript is stored inside a Web Site:
      
        Traceback (innermost last):
        Module Products.CMFActivity.ActivityTool, line 1360, in invokeGroup
          traverse(method_id)(expanded_object_list)
        Module Products.ERP5Catalog.CatalogTool, line 939, in catalogObjectList
          super(CatalogTool, self).catalogObjectList(tmp_object_list, **m.kw)
        Module Products.ZSQLCatalog.ZSQLCatalog, line 828, in catalogObjectList
          catalog_value=catalog,
        Module Products.ERP5Catalog.CatalogTool, line 875, in wrapObjectList
          - __traceback_info__: (<PythonScript at /erp5/web_site_module/...>,)
          and document_object._getAcquireLocalRoles():
        AttributeError: _getAcquireLocalRoles
      
      Before, such object was skipped because it does not have recursiveReindexObject.
      614a8349
  11. 04 Jul, 2018 1 commit
    • Vincent Pelletier's avatar
      all: Remove recursiveImmediateReindexObject, and references to it. · c0b284be
      Vincent Pelletier authored
      This method does not belong to Folder (nor Base, even as an alias), as
      it recurses wihout any limit in a tree whose node count is unknown.
      Such generic recursion should either be deferred (split in multiple
      transactions, each processing a limited chunk of the tree) or not be
      recursive (which of course depends on the need).
      There is a single caller left (constructInstance) where the subtree size
      is assumed to be reasonable, as it was constructed inside current
      transaction. So essentially rename this method so it is only used in such
      conditions.
      c0b284be
  12. 03 Jul, 2018 4 commits
    • Vincent Pelletier's avatar
      Base: Add isSubtreeIndexable and isAncestryIndexable · 5c4c69a0
      Vincent Pelletier authored
      isAncestryIndexable is a parent-recursive version of isIndexable: it
      will return False if any parent is non-indeable. This simplifies
      disabling indexation of an entire tree, from its root to its deepest
      leaves.
      isSubtreeIndexable allows preventing any subobject from being indexable,
      without having to mark each individual subobject as non-indexable. This
      is different from isAncestryIndexable, as it allows having an indexable
      root, but no children indexable. This is especially useful for Trash Bins,
      where the bin itself should be indexable (because there is no reason not to)
      but its content must not be indexed, while being left as it was found in its
      original location.
      5c4c69a0
    • Vincent Pelletier's avatar
      Folder: Make recursiveReindexObject scalable by calling _recurseCallMethod. · babbf56f
      Vincent Pelletier authored
      Should make Folder_reindexAll and most custom indexation methods obsolete.
      Remaining valid reindexation methods are:
      - reindexObject: for a single document, which may contain subdocuments
        which indexation is not necessary
      - recursiveReindexobject: for any subtree of documents
      - ERP5Site_reindexAll: for site-wide reindexations, as there is a semantic-
        dependent indexation order.
      
      Also, uniformise and factorise spawning immediateReindexObject.
      
      Also:
      - testSupply: Drop check for the previous magic threshold.
        _recurseCallMethod takes care of it all now.
      - testXMLMatrix: Let activities execute before changing cell id.
        This works only because recursiveReindexObject on the matrix spawns a
        single recursiveImmediateReindexObject activity on that context. Now,
        up to 1k immediateReindexObject activities (for the first 1k sub-objects)
        are spawned immediately, preventing their renaming immediately after
        commit.
        So let test wait for indexation before trying to rename.
      - testERP5Security: More activities are now spawned immediately, adapt.
      babbf56f
    • Vincent Pelletier's avatar
      EPR5Type.Core.Folder: Actually add support for OFS.Folder. · b9202251
      Vincent Pelletier authored
      So it is not just an unused constant (OFS_HANDLER) and no implementation.
      b9202251
    • Vincent Pelletier's avatar
      ERP5Type.Core.Folder: Simplify proxy methods. · 8dcfdf4f
      Vincent Pelletier authored
      Also, clarify what exactly was the broken BTree marker value, fixing
      isBTree accessor. Update associated tests.
      8dcfdf4f
  13. 26 Apr, 2018 1 commit
  14. 27 Mar, 2018 2 commits
  15. 09 Mar, 2018 4 commits
  16. 02 Feb, 2018 1 commit
    • Vincent Pelletier's avatar
      ERP5Type.Core.Folder.recurseCallMethod: Do not change priority for re-call · 24df3ec5
      Vincent Pelletier authored
      Increasing priority value (hence making the activity execution priority
      lower) means that all previously-spawned activities will have at least
      started execution before the next batch of activities begins being
      prepared. During that preparation, more activities will end, meaning there
      is then unused processing node time, decreasing processing efficiency.
      
      By making re-call activity have the same priority as spawned activities
      will reduce this delay by making the re-call activity candidate for
      execution at the same time as the latest activity batch, without leading
      to activity queue size increase: older activities within a given priority
      range still get precedence, so any older group will tend to finish before
      more batches get spawned.
      
      Measures on indexing 700k documents with 20 activity nodes:
      - priority + 1: 84k documents indexed in the first 5 minutes,
        47k in the last 5 minutes. Average: 64k/5 minutes
      - priority: 101k documents indexed in the first 5 minutes,
        97k in the last 5 minutes. Average: 99k/5 minutes
      Peak speed (which is the initial speed) is 20% higher.
      As the slowdown rate (likely coming from lower-priority activities piling
      up in the message_queue table) is also much lower in the second case
      (900 fewer documents indexed on each 5 minutes increment instead of 4k),
      the overall average speed improves by 54%.
      24df3ec5
  17. 21 Nov, 2017 2 commits
  18. 06 Nov, 2017 2 commits
  19. 20 Oct, 2017 1 commit
  20. 26 Sep, 2017 1 commit
  21. 02 Sep, 2016 1 commit
  22. 15 Jul, 2016 1 commit
  23. 13 Feb, 2016 1 commit
  24. 12 Jan, 2016 1 commit
  25. 24 Nov, 2015 1 commit
  26. 16 Dec, 2014 1 commit
  27. 18 Nov, 2014 2 commits