1. 24 May, 2016 3 commits
    • Vincent Pelletier's avatar
      CatalogTool: Assorted readability improvements. · da29e93c
      Vincent Pelletier authored
      Fix a misnamed local.
      Use iteritems.
      Move variables next to the loop they act on.
      Rewrite comments to better reflect the intention when it is not obvious in
      the implementation.
      Move a loop-invariant out of loop ((user, role) not in optimized_role_set).
      Do not use set.update on an iterable constructed just for the call, call
      set.add on each item.
    • Vincent Pelletier's avatar
      CatalogTool: Avoid computing user's roles when just testing existence. · ed0c06f9
      Vincent Pelletier authored
      Global role computation is very expensive, don't do it when all we are
      interested in is user existence.
      Also, list all users having local roles in one call to reduce the effect
      of per-call overhead.
      This should improve indexation speed by a large fraction: pprofile
      statistical run on and activity node busy reindexing objects shows:
      - 57% of time spend generating (13%) and executing (44%) SQL
      - 42% of time spent preparing object for indexation:
        - 29% in _getSecurityParameterList, all in getUserById
        - 12% in getCatalogVariablesFor (workflow variables to index)
      So an improvement of close to 30% is expected.
    • Vincent Pelletier's avatar
      CatalogTool: Move getUserById call up one loop. · 6be1d3a1
      Vincent Pelletier authored
      Its parameter is a loop invariant, so avoid looking the same user up once
      per local role.
  2. 12 Jan, 2016 1 commit
  3. 24 Sep, 2015 1 commit
  4. 23 Sep, 2015 2 commits
  5. 14 Aug, 2015 2 commits
  6. 10 Jul, 2015 1 commit
    • Vincent Pelletier's avatar
      EP5Type.Utils: Update SQL escaping rules. · 39c4873a
      Vincent Pelletier authored
      Sadly, we still implement our own escaping, as places escaping strings do
      not know which connector will be used (proper escaping is
      connector-dependent, because database-dependent).
      Move this method in ZSQLCatalog to factorise code.
  7. 10 Jun, 2015 1 commit
  8. 13 May, 2015 2 commits
    • Julien Muchembled's avatar
      searchAndActivate: use a grouping method instead of callMethodOnObjectList · 3759047d
      Julien Muchembled authored
      The problem with callMethodOnObjectList is that when an object can't be
      processed, all other objects of the same group fail without any chance to
      be retried separately.
      Grouping is configurable with usual CMFActivity parameters in new 'group_kw'
      parameter, to avoid any conflict with catalog parameters (**kw).
      'packet_size' and 'activity_count' are still accepted for backward
    • Julien Muchembled's avatar
      CMFActivity: new invokeGroup API · da234001
      Julien Muchembled authored
      The recent API change was not enough.
      A grouping method may need more information: in particular, the dummy grouping
      method must be fixed to change user.
  9. 06 May, 2015 1 commit
    • Julien Muchembled's avatar
      CMFActivity: slightly delay non-executed grouped messages · c85a840f
      Julien Muchembled authored
      When grouped messages fail, ActivityTool must distinguish 3 groups,
      in order to reexecute them separately, as follows:
      - first, those that succeeded
      - then, those that were skipped
      - at last, failed ones
      Grouping methods are updated to handle partial failures, and stop doing
      anything when something goes wrong.
      Without this, we would have the following pathological cases.
      1. Let's suppose first that skipped messages are marked as succeeded.
      The problem is that each skipped message that will fail causes the reexecution
      of those that didn't fail.
      Exemple: A:ok B:ok C:err D:err E:err F:err
        1: A:ok, B:ok, C:err, D:skipped, E:skipped, F:skipped
        2: A:ok, B:ok, D:err, E:skipped, F:skipped
        3: A:ok, B:ok, E:err, F:skipped
        4: A:ok, B:ok, F:err
        5: A:ok, B:ok -> commit
      And worst, the first failed (C) may be processable again before 5, entering
      a failing loop if it is executed again in the same group as A & B.
      2. Another implementation is to mark all skipped as failed.
        1: A:ok, B:ok, C:err, D:skipped, E:skipped, F:skipped
        2: A:ok, B:ok -> commit
        3: C:err, D:skipped, E:skipped, F:skipped
       >3: same as 3
      => D, E or F are never tried.
  10. 31 Mar, 2015 1 commit
  11. 25 Mar, 2015 1 commit
  12. 29 Jan, 2015 1 commit
  13. 08 Nov, 2014 1 commit
    • Kazuhiko Shiozaki's avatar
      use fulltext search in title and description. · d47df833
      Kazuhiko Shiozaki authored
      * to quickly setup catalog_full_text table, you can use the following SQL.
        REPLACE INTO catalog_full_text SELECT uid, title, description FROM catalog;
      * non fulltext queries like '=abc', '>abc', '%abc%' are supported.
      * now erp5_full_text_mroonga_catalog is used for unit tests thus I recommend using it instead of erp5_full_text_myisam_catalog.
      * to migrate existing MyISAM full_text table into Mroonga, you can use the following SQL.
        ALTER TABLE full_text DROP KEY SearchableText,
          ENGINE = mroonga,
          ADD FULLTEXT KEY SearchableText (`SearchableText`) COMMENT 'parser "TokenBigramSplitSymbolAlpha"';
      * fulltext search score is no longer provided as (column_name) but now provided as (column_name)__score__.
      * (category)_title, like source_title, related keys are automatically generated. (category)_description keys as well.
  14. 16 Oct, 2014 1 commit
  15. 08 Sep, 2014 2 commits
  16. 04 Sep, 2014 1 commit
  17. 30 Apr, 2014 3 commits
  18. 19 Mar, 2014 1 commit
    • Julien Muchembled's avatar
      CatalogTool: fix searchAndActivate when called with a condition on 'uid' · 25ff6cf0
      Julien Muchembled authored
      Because the method internally overrode 'uid' after the first group of objects,
      passing a list of uids could make it go berserk and activate all objects in
      catalog with an uid greater than those of the first group.
      Although searchAndActivate was not initially written to be called with such
      list, this commit fixes this instead of forbidding conditions on 'uid',
      since it remains useful to easily split & group activities.
  19. 10 Mar, 2014 3 commits
  20. 21 Jan, 2014 1 commit
  21. 19 Aug, 2013 1 commit
  22. 23 May, 2013 1 commit
  23. 27 Mar, 2013 1 commit
    • Vincent Pelletier's avatar
      getSecurityUidDictAndRoleColumnDict micro-optimisation · c70f1f3d
      Vincent Pelletier authored
      try..except is significantly slower than getattr with default value when
      an exception is raised.
      defaultdict is faster than calling setdefault.
      The two other try..except blocks need more careful analysis of typical
      hit-rates to tell if they are optimal.
  24. 12 Jul, 2012 5 commits
  25. 27 Jun, 2012 1 commit
  26. 22 Jun, 2012 1 commit