1. 01 May, 2017 4 commits
    • Jérome Perrin's avatar
      upgrader: upgrade catalog schema right after installing BTs · ba541ad4
      Jérome Perrin authored
      93e30e5e introduce a  new `user` table that is listed as a *search table*, so until this table is created all catalog queries are failing, and alarm tool, which relies on catalog fail with such a traceback:
      
      ```
      ERROR TimerService Process timer error
      Traceback (most recent call last):
        File "parts/erp5/product/TimerService/TimerService.py", line 102, in process_timer
          DateTime(prev_tick), DateTime(next_tick))
        File "parts/erp5/product/ERP5/Tool/AlarmTool.py", line 175, in process_timer
          self.tic()
        File "parts/erp5/product/ERP5/Tool/AlarmTool.py", line 135, in tic
          for alarm in self.getAlarmList(to_active=1):
        File "parts/erp5/product/ERP5/Tool/AlarmTool.py", line 111, in getAlarmList
          alarm_date={'query':now,'range':'ngt'}
        File "parts/erp5/product/ERP5Catalog/CatalogTool.py", line 702, in unrestrictedSearchResults
          return ZCatalog.searchResults(self, **kw)
        File "parts/erp5/product/ZSQLCatalog/ZSQLCatalog.py", line 1091, in searchResults
          return catalog.searchResults(REQUEST, **kw)
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 2585, in searchResults
          **kw
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 2554, in queryResults
          **kw
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 2418, in buildSQLQuery
          ignore_unknown_columns=ignore_unknown_columns,
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 2394, in buildEntireQuery
          query=self.buildQuery(kw, ignore_empty_string=ignore_empty_string, ignore_unknown_columns=ignore_unknown_columns),
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 2295, in buildQuery
          result = self.buildSingleQuery(key, value)
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 2087, in buildSingleQuery
          search_key, related_key_definition = self.getColumnSearchKey(key, search_key_name)
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 2049, in getColumnSearchKey
          related_key_definition = self.getRelatedKeyDefinition(key)
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 1999, in getRelatedKeyDefinition
          for entire_definition in self.getSQLCatalogRelatedKeyList([key]):
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 1935, in getSQLCatalogRelatedKeyList
          column_map = self._getSQLCatalogRelatedKeySet()
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 130, in wrapper
          result = transactional_cache[cache_id] = method(wrapped_self)
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 1908, in _getSQLCatalogRelatedKeySet
          column_map = self.getColumnMap()
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 130, in wrapper
          result = transactional_cache[cache_id] = method(wrapped_self)
        File "parts/erp5/product/ZSQLCatalog/SQLCatalog.py", line 1146, in getColumnMap
          for field in table_dict[table]:
      KeyError: 'user'
      ```
      
      This means that even if upgrader's post upgrade constraint are supposed to fix this by calling `portal_catalog.upgradeSchema`, because this constraint rely on alarm tool, it's already too late.
      
      The suggested way to fix this is to also call [`portal_catalog.upgradeSchema`](https://lab.nexedi.com/nexedi/erp5/blob/c99fb9163503c5afdef59ecb124b3060b05330a4/bt5/erp5_upgrader/SkinTemplateItem/portal_skins/erp5_upgrader/TemplateTool_checkTableConsistency.py#L7) in the same transaction that the transaction upgrading business templates.
      
      I have not completely removed `TemplateToolTableConsistencyConstraint`, because maybe there will be cases where just call post upgrade alarm to execute data migration steps after installing a business template manually. But you guys think it's better to completely merge `TemplateToolTableConsistencyConstraint` with `TemplateToolBusinessTemplateInstallationConstraint` I would be OK with that too.
      
      Of course, we cannot just make
      `TemplateToolTableConsistencyConstraint` an `upgrade` constraint, because there would be no guarantee that it's called after `TemplateToolBusinessTemplateInstallationConstraint`.
      
      Also, I have not considered making [`getColumnMap` and friends](https://lab.nexedi.com/nexedi/erp5/blob/master/product/ZSQLCatalog/SQLCatalog.py#L1142) resilient to the case where a table listed in search tables does not exist, because it's would just have been failing later anyway and it's better to fail early in such case.
      
      cc: @vpelletier @jm @seb @tiwariayush @gabriel
      
      /reviewed-on nexedi/erp5!247
      ba541ad4
    • Jérome Perrin's avatar
      Lower priority of mass workflow transition activity · f88d9684
      Jérome Perrin authored
      We had an incident in a in instance were a user changed state of 70K invoices using https://www.erp5.com/howto/erp5-developer-howto/erp5-HowTo.Change.Workflow.State.Of.Multiple.Documents  and leads to user not receiving the reports they requested.
      
      `Folder_modifyWorkflowStatus` creates a lot of `callMethodOnObjectList` activities (for all the selected documents) with priority 2, then these activities will cause more reindex an expand activities.
      
      Until all these `callMethodOnObjectList` are processed, no new activities with priority > 2 were processed. Some "important for users" activities such as erp5_deferred_style reports where waiting in the queue.
      
      I believe we should just set a lower priority to these `callMethodOnObjectList`, eventhough I considered a more clever way of giving the priority:
      
       - if the number of selected documents is reasonably small, process them with a very high priority, this way the user can see his document changing state almost immediately as when using the synchronous change state. This should not cause congestion because there are not too many documents.
       - when there are a lot of selected documents, process them with a very low priority, because anyway it will take time and user will not "wait" for documents to change state.
      
      ... but I realized this is trying to be too clever.
      
      So any objections to just lower priority here ?
      
      /reviewed-on nexedi/erp5!235
      f88d9684
    • Jérome Perrin's avatar
      Support arbitrary email headers · cb9d0c70
      Jérome Perrin authored
      As discussed in  nexedi/erp5!248 this approach allows to set any mail header.
      
      I also included a not so related patch of email header handlings 88d40b40 so that we review all this together.
      
      /cc @gabriel @kazuhiko 
      
      /reviewed-on nexedi/erp5!256
      cb9d0c70
    • Jérome Perrin's avatar
      Enable erp5 login user manager in post_upgrade · 8a27134c
      Jérome Perrin authored
      I just simulated an upgrade for an ERP5 running a version from before !185,  `ERP5UserManager Non Existence_constraint` properly disabled old `acl_users/erp5_users` and migrated all persons, but  `ERP5UserLoginManager Existence_constraint` did not activate `acl_users/erp5_login_users`.
      
      This is because  `ERP5UserLoginManager Existence_constraint` is registered as a **pre-upgrade** constraint, but with my upgrade scenario, pre-upgrade constraint are installed too late. 
      
      The upgrade scenario was:
      
       1. install new ERP5 SR on slapos
       2. manually install new versions of  erp5_upgrader and customer_configuration_upgrader business template
       3. run upgrade ( sense then fix on portal_alarms/promise_check_upgrade )
      
      During step 3, when pre-upgrade constraints are executed, erp5_base is not installed yet, so  `ERP5UserLoginManager Existence_constraint` can not run because it's not installed yet. 
      
      My suggestion is to change it to a post-upgrade constraint.
      
      If I understand correctly and my upgrade scenario is really the supported one, it means we can only have pre-upgrade constraints in erp5_upgrader.
      
      ( I also fix a meaningless typo in the same code)
      
      /cc @kazuhiko  @vpelletier @seb @gabriel 
      
      /reviewed-on nexedi/erp5!262
      8a27134c
  2. 28 Apr, 2017 2 commits
  3. 27 Apr, 2017 1 commit
  4. 25 Apr, 2017 2 commits
  5. 24 Apr, 2017 1 commit
    • Vincent Pelletier's avatar
      ERP5Security.ERP5UserLoginManager: Special-case user_id='System Processes' · 17d3df41
      Vincent Pelletier authored
      Because of ERP5Type.UnrestrictedMethod, 'System Processes' can own objects.
      Such objects can be proxy-role'd scripts, and proxy-role mechanism
      triggers many users look-ups (each time security is evaluated, which is
      virtually every getattr). Each such lookup will do a query for 'System
      Processes' user, which will (hopefully) find nothing anyway.
      So special-case 'System Processes' when looking by user_id by skipping
      the search altogether (enforcing the inability to locate this user,
      consistently with Zope assumptions, and consistently with previous
      behaviour).
      17d3df41
  6. 21 Apr, 2017 4 commits
  7. 20 Apr, 2017 14 commits
  8. 19 Apr, 2017 2 commits
  9. 18 Apr, 2017 10 commits