upgrader: really execute the whole `upgrade` step in one transaction

Documentation says this step is done in one transaction, so implement
this as documented.

Detecting developer mistake and automagically running this step in smaller
transactions if more than 100 documents seems unnecessary. At least, it
contredicts documentation.

bt5/erp5_upgrader/SkinTemplateItem/portal_skins/erp5_upgrader/Alarm_runUpgrader.py

@@ -3,12 +3,9 @@
 
   IMPORTANT: Don't use the constraint_type upgrader to data migration or big amount of objects,
   because this step is suppose to run all constraints in the same transaction. 
-  To not kill the instance, searchAndActivate will be used if countResults() > REINDEX_SPLIT_COUNT
 """
 
-REINDEX_SPLIT_COUNT = 100
 portal = context.getPortalObject()
-portal_alarms = portal.portal_alarms
 
 _, type_per_constraint_type = context.Base_getConstraintTypeListPerPortalType()
 portal_type_list = type_per_constraint_type.get('upgrader', [])
@@ -27,17 +24,11 @@ with context.defaultActivateParameterDict(activate_kw, placeless=True):
     'active_process': active_process.getRelativeUrl(),
     'activate_kw': activate_kw,
   }
-
+  # always run on portal_templates, regardless of catalog state.
   portal.portal_templates.Base_postCheckConsistencyResult(**method_kw)
-  for portal_type in portal_type_list:
-    if portal.portal_catalog.countResults(
-        portal_type=portal_type_list)[0][0] > REINDEX_SPLIT_COUNT:
-      portal.portal_catalog.searchAndActivate('Base_postCheckConsistencyResult',
-        activate_kw=activate_kw,
-        portal_type=portal_type,
-        method_kw=method_kw)
-    else:
-      for result in portal.portal_catalog(portal_type=portal_type):
+  # run on all portal_types with an `upgrader` constraint, except portal_templates we already run before.
+  if portal_type_list:
+    for result in portal.portal_catalog(portal_type=portal_type_list):
       result.Base_postCheckConsistencyResult(**method_kw)
 
 context.setEnabled(False)

bt5/erp5_upgrader/TestTemplateItem/portal_components/test.erp5.testUpgrader.py

@@ -343,28 +343,12 @@ class TestUpgrader(ERP5TypeTestCase):
     self.assertEqual({'immediateReindexObject', 'unindexObject'},
       {x.method_id for x in portal_activities.getMessageList()})
 
-  def stepCreateBigIncosistentData(self, sequence=None):
-    for _ in range(101):
-      self.portal.organisation_module.newContent(
-        portal_type="Organisation",
-        title="org_%s" % self.portal.organisation_module.getLastId())
-
   def stepCreateSmallIncosistentData(self, sequence=None):
     for _ in range(4):
       self.portal.organisation_module.newContent(
         portal_type="Organisation",
         title="org_%s" % self.portal.organisation_module.getLastId())
 
-  def stepCheckActivitiesCreated(self, sequence=None):
-    portal_activities = self.getActivityTool()
-    self.assertEqual({'Alarm_runUpgrader', 'notify'},
-      {x.method_id for x in portal_activities.getMessageList()})
-    for message in portal_activities.getMessageList():
-      portal_activities.manageInvoke(message.object_path, message.method_id)
-    self.commit()
-    self.assertIn('Base_postCheckConsistencyResult',
-      {x.method_id for x in portal_activities.getMessageList()})
-
   def stepUninstallERP5UpgraderTestBT(self, sequence=None):
     bt5 = self.portal.portal_templates.getInstalledBusinessTemplate('erp5_web')
     bt5.uninstall()
@@ -656,12 +640,6 @@ class TestUpgrader(ERP5TypeTestCase):
       stepTic
       stepRunUpgrader
       stepCheckNoActivitiesCreated
-      stepCreateBigIncosistentData
-      stepTic
-      stepActiveSenseUpgradeAlarm
-      stepTic
-      stepRunUpgrader
-      stepCheckActivitiesCreated
       stepRemoveConstraintFromOrganisationPortalType
     """
     sequence_list.addSequenceString(sequence_string)

[Sebastien Robin]

I fully agree with this. The step "upgrade" must not be splitted. It contains all steps that must be done in one transaction to avoid having inconsistent and broken status. The idea is to do upgrades with no downtime at all. And some steps can not be separated (like we already merged the installation of all business templates to avoid inconsistencies). If migration of many objects is wished, it must be done in before or after upgrade, or it should be handled by automatic migration as soon as objects are accessed.