Commit 5c09e2e2 by Julien Muchembled

Simulation: splitted expand, performance improvements and bugfixes

All interactions and activity tags are reviewed to fix bugs like duplicated
root applied rules, and also reduces the amount of duplicated/useless work, e.g:
- Simulation trees are not expanded anymore when simulated objects are modified.
- 'expand' activities are merged (i.e. dropped) with any other 'expand' activity
  for an ancestor.
New implementation exposes new API that hides much complexity to the developper
about activity dependencies.

By default, expand() now automatically defers any work if the current
transaction takes too long time. This method also gains a parameter to
explicitely choose when to expand, which is often important in unit tests or
solvers. In particular, when postponing work, it takes care of setting proper
activity dependencies.
- If you have any code requiring to expand everything immediately, you'll have
  to replace 'expand()' by 'expand(expand_policy="immediate")'.
- On the contrary, you should replace any 'activate().expand()' by
  'expand(expand_policy="deferred")'.
expand() still accepts activity parameters for any extra needs.

In causality workflow, 'building' state is clarified and now means
« delivery may diverge but we can't know now ». A delivery remains in draft
as long as it does not contain any movement built from simulation.
After init/clone/builder/etc. scripts used to call 'startBuilding' &
'updateCausalityState': this calls must be removed since only
SimulatedDeliveryBuilder should take care of move to 'building' state and
workflows now triggers 'updateCausalityState'.

Disguised interactions have been unhardcoded and either deleted, or moved to
appropriate interaction workflows, which have been reorganized. Those
that triggers update of portal_workflow can be easily customized or disabled.

New API:
- updateSimulation() on deliveries and subscription items. It takes care of
  creating root applied rule, expanding and reindexing parts of simulation
  trees. It somehow replaces:
  - Delivery_updateSimulation
  - Delivery_updateAppliedRule
  - Delivery.applyToDeliveryRelatedMovement
  - Delivery.updateAppliedRule
  - Delivery.expand
  - Delivery.expandRuleRelatedToMovement
  - SubscriptionItem.expand
  - SubscriptionItem.updateAppliedRule
- Delivery.localBuild() is the new way to do local building and replaces
  Delivery_expandAndBuild. Private method Delivery._localBuild replaces
  Delivery_buildOnComposedDocument.
- Simulation Movements that are being built by a builder are reindexed with
  the following tag: 'built:<delivery_path>'. Any after_path_and_method_id
  dependency against 'related_simulation_movement_path_list' and reindexing
  methods should be replaced by this after_tag.

After builder scripts used to confirm the delivery in a separate activity,
which was useless.
1 parent fb246bec
Showing 173 changed files with 171 additions and 3016 deletions
......@@ -56,11 +56,6 @@ context.setSourceReference(None)\n
context.setReference(None)\n
context.setDestinationReference(None)\n
context.setSolver(None)\n
\n
# Initialize Causality Workflow\n
if hasattr(context, \'startBuilding\'):\n
context.startBuilding()\n
context.updateCausalityState()\n
</string> </value>
</item>
<item>
......
......@@ -59,7 +59,7 @@
</item>
<item>
<key> <string>id</string> </key>
<value> <string>AccountingRuleCell_getRuleReference</string> </value>
<value> <string>AccountingTransaction_getRuleReference</string> </value>
</item>
</dictionary>
</pickle>
......
......@@ -50,23 +50,15 @@
</item>
<item>
<key> <string>_body</string> </key>
<value> <string>from Products.CMFCore.utils import getToolByName\n
\n
transaction = context\n
preference_tool = getToolByName(context, \'portal_preferences\')\n
\n
transaction.edit (\n
<value> <string>preference_tool = context.getPortalObject().portal_preferences\n
context.edit(\n
source_section = preference_tool.getPreferredAccountingTransactionSourceSection(),\n
resource = preference_tool.getPreferredAccountingTransactionCurrency())\n
\n
if hasattr(transaction, \'startBuilding\') :\n
transaction.startBuilding()\n
transaction.updateCausalityState()\n
</string> </value>
</item>
<item>
<key> <string>_params</string> </key>
<value> <string>*args, **kw</string> </value>
<value> <string></string> </value>
</item>
<item>
<key> <string>id</string> </key>
......
......@@ -59,7 +59,7 @@
</item>
<item>
<key> <string>id</string> </key>
<value> <string>PurchaseInvoiceTransaction_getRuleReference</string> </value>
<value> <string>Invoice_getRuleReference</string> </value>
</item>
</dictionary>
</pickle>
......
......@@ -57,13 +57,6 @@ payment_transaction = context\n
# initialize accounting_workflow to planned state\n
if payment_transaction.getSimulationState() == "draft":\n
payment_transaction.plan(comment=translateString("Initialised by Delivery Builder."))\n
\n
# First set the payment transaction in the building state on the causality workflow\n
payment_transaction.startBuilding()\n
\n
# Then an activity should put the causality state in diverged or solved\n
payment_transaction.activate(after_path_and_method_id=(related_simulation_movement_path_list,\n
(\'immediateReindexObject\',\'recursiveImmediateReindexObject\'))).updateCausalityState()\n
</string> </value>
</item>
<item>
......
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="PythonScript" module="Products.PythonScripts.PythonScript"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>Script_magic</string> </key>
<value> <int>3</int> </value>
</item>
<item>
<key> <string>_bind_names</string> </key>
<value>
<object>
<klass>
<global name="NameAssignments" module="Shared.DC.Scripts.Bindings"/>
</klass>
<tuple/>
<state>
<dictionary>
<item>
<key> <string>_asgns</string> </key>
<value>
<dictionary>
<item>
<key> <string>name_container</string> </key>
<value> <string>container</string> </value>
</item>
<item>
<key> <string>name_context</string> </key>
<value> <string>context</string> </value>
</item>
<item>
<key> <string>name_m_self</string> </key>
<value> <string>script</string> </value>
</item>
<item>
<key> <string>name_subpath</string> </key>
<value> <string>traverse_subpath</string> </value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</state>
</object>
</value>
</item>
<item>
<key> <string>_body</string> </key>
<value> <string>return \'default_invoice_rule\'\n
</string> </value>
</item>
<item>
<key> <string>_params</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>SaleInvoiceTransaction_getRuleReference</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="PythonScript" module="Products.PythonScripts.PythonScript"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>Script_magic</string> </key>
<value> <int>3</int> </value>
</item>
<item>
<key> <string>_bind_names</string> </key>
<value>
<object>
<klass>
<global name="NameAssignments" module="Shared.DC.Scripts.Bindings"/>
</klass>
<tuple/>
<state>
<dictionary>
<item>
<key> <string>_asgns</string> </key>
<value>
<dictionary>
<item>
<key> <string>name_container</string> </key>
<value> <string>container</string> </value>
</item>
<item>
<key> <string>name_context</string> </key>
<value> <string>context</string> </value>
</item>
<item>
<key> <string>name_m_self</string> </key>
<value> <string>script</string> </value>
</item>
<item>
<key> <string>name_subpath</string> </key>
<value> <string>traverse_subpath</string> </value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</state>
</object>
</value>
</item>
<item>
<key> <string>_body</string> </key>
<value> <string>delivery = state_change[\'object\']\n
delivery.Delivery_expandAndBuild()\n
</string> </value>
</item>
<item>
<key> <string>_params</string> </key>
<value> <string>state_change</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>Delivery_Build</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
......@@ -50,7 +50,7 @@
</item>
<item>
<key> <string>_body</string> </key>
<value> <string>state_change[\'object\'].Delivery_updateSimulation()\n
<value> <string>state_change[\'object\'].localBuild()\n
</string> </value>
</item>
<item>
......@@ -59,7 +59,7 @@
</item>
<item>
<key> <string>id</string> </key>
<value> <string>Delivery_updateSimulation</string> </value>
<value> <string>Delivery_localBuild</string> </value>
</item>
</dictionary>
</pickle>
......
......@@ -24,7 +24,7 @@
</item>
<item>
<key> <string>after_script_name</string> </key>
<value> <string>Delivery_Build</string> </value>
<value> <string>Delivery_localBuild</string> </value>
</item>
<item>
<key> <string>description</string> </key>
......
......@@ -11,6 +11,10 @@
<value> <string>workflow</string> </value>
</item>
<item>
<key> <string>actbox_icon</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>actbox_name</string> </key>
<value> <string></string> </value>
</item>
......@@ -20,7 +24,7 @@
</item>
<item>
<key> <string>after_script_name</string> </key>
<value> <string>Delivery_updateSimulation</string> </value>
<value> <string></string> </value>
</item>
<item>
<key> <string>description</string> </key>
......
......@@ -11,6 +11,10 @@
<value> <string>workflow</string> </value>
</item>
<item>
<key> <string>actbox_icon</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>actbox_name</string> </key>
<value> <string></string> </value>
</item>
......@@ -20,7 +24,7 @@
</item>
<item>
<key> <string>after_script_name</string> </key>
<value> <string>Delivery_Build</string> </value>
<value> <string>Delivery_localBuild</string> </value>
</item>
<item>
<key> <string>description</string> </key>
......
......@@ -37,6 +37,7 @@
<key> <string>tested_property</string> </key>
<value>
<tuple>
<string>specialise</string>
<string>source_section</string>
<string>destination_section</string>
<string>source_payment</string>
......
......@@ -37,6 +37,7 @@
<key> <string>tested_property</string> </key>
<value>
<tuple>
<string>specialise</string>
<string>source_section</string>
<string>destination_section</string>
<string>source_payment</string>
......
......@@ -71,9 +71,11 @@ for builder_id in builder_id_list:\n
serialization_tag = \'build:%s\' % delivery_portal_type\n
index_tag = \'index:%s\' % delivery_portal_type\n
after_tag = index_tag\n
# depend on reindexing so that select methods\n
# do not return movements that are already built\n
after_method_id = (\'recursiveImmediateReindexObject\',\n
\'immediateReindexObject\',\n
\'Delivery_updateAppliedRule\')\n
\'_updateSimulation\')\n
activate_kw = dict(tag=index_tag)\n
builder.activate(\n
serialization_tag=serialization_tag,\n
......
......@@ -54,14 +54,6 @@
the new Invoice.\n
"""\n
from Products.ERP5Type.Message import translateString\n
try:\n
from Products.CMFCore.WorkflowCore import WorkflowException\n
except ImportError:\n
# WorkflowException has not always been allowed in restricted\n
# environment, in this case, make sure WorkflowException is \n
# defined \n
class WorkflowException(Exception):\n
pass\n
\n
if related_simulation_movement_path_list is None:\n
raise RuntimeError, \'related_simulation_movement_path_list is missing. Update ERP5 Product.\'\n
......@@ -80,26 +72,10 @@ if not invoice.hasTitle() and related_packing_list.hasTitle():\n
\n
# initialize accounting_workflow to confirmed state\n
if invoice.getSimulationState() == \'draft\':\n
try :\n
context.getPortalObject().portal_workflow.doActionFor(\n
invoice, \'confirm_action\',\n
comment=translateString(\'Initialised by Delivery Builder.\'),\n
skip_period_validation=1)\n
except WorkflowException, e:\n
# The user cannot pass the transition, it\'s OK\n
pass\n
\n
if invoice.getSimulationState() == \'draft\':\n
# call the workflow method, if the user cannot perform this operation.\n
invoice.confirm(comment=translateString(\'Initialised by Delivery Builder.\'),)\n
\n
\n
# First set the invoice in the building state on the causality workflow\n
invoice.startBuilding()\n
\n
# Then an activity should put the causality state in diverged or solved\n
invoice.activate(after_path_and_method_id=(related_simulation_movement_path_list,\n
(\'immediateReindexObject\',\'recursiveImmediateReindexObject\'))).updateCausalityState()\n
invoice.Delivery_confirm()\n
else:\n
# call builder just same as after script of \'confirm\' transition\n
invoice.localBuild()\n
</string> </value>
</item>
<item>
......
......@@ -50,64 +50,12 @@
</item>
<item>
<key> <string>_body</string> </key>
<value> <string>"""\n
Builds the delivery.\n
"""\n
from Products.ERP5Type.Log import log\n
delivery = sci[\'object\']\n
delivery_portal_type = delivery.getPortalType()\n
portal_deliveries = sci.getPortal().portal_deliveries\n
\n
builder_by_ptype = {\n
\'Sale Invoice\' : \'advanced_sale_invoice_transaction_builder\',\n
\'Purchase Invoice\' : \'advanced_purchase_invoice_transaction_builder\',\n
}\n
\n
if builder_by_ptype.has_key(delivery_portal_type) :\n
builder = getattr(portal_deliveries, builder_by_ptype[delivery_portal_type], None)\n
if builder is None :\n
log(\'erp5_advanced_invoicing\',\n
\'unable to build : no builder in %s\' % builder_by_ptype[delivery_portal_type])\n
return\n
\n
# build accounting lines\n
method_id_list = (\'expand\', \'edit\', \'updateAppliedRule\', \'Delivery_updateAppliedRule\',\n
\'immediateReindexObject\', \'recursiveImmediateReindexObject\')\n
\n
explanation_uid_list = [delivery.getUid(), ]\n
packing_list = delivery.getCausalityValue(\n
portal_type=(\'Sale Packing List\',\n
\'Purchase Packing List\'))\n
if packing_list is not None:\n
explanation_uid_list.append(packing_list.getUid())\n
order = packing_list.getCausalityValue(\n
portal_type=(\'Sale Order\',\n
\'Purchase Order\'))\n
if order is not None:\n
explanation_uid_list.append(order.getUid())\n
\n
\n
tag = \'invoice_transaction_build_%s\' % delivery.getRelativeUrl()\n
builder.activate(\n
activity=\'SQLQueue\',\n
after_method_id=method_id_list,\n
tag=tag,\n
activate_kw=dict(tag=tag)).build(activate_kw=dict(tag=tag),\n
explanation_uid=explanation_uid_list)\n
\n
# build related payment transactions\n
portal_deliveries.payment_transaction_builder.activate(\n
activity=\'SQLQueue\',\n
after_method_id=method_id_list).build(explanation_uid=explanation_uid_list)\n
\n
# set the object in building state.\n
delivery.startBuilding()\n
delivery.activate(after_tag=tag).updateCausalityState()\n
<value> <string>state_change[\'object\'].localBuild()\n
</string> </value>
</item>
<item>
<key> <string>_params</string> </key>
<value> <string>sci</string> </value>
<value> <string>state_change</string> </value>
</item>
<item>
<key> <string>id</string> </key>
......
......@@ -11,6 +11,10 @@
<value> <string>workflow</string> </value>
</item>
<item>
<key> <string>actbox_icon</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>actbox_name</string> </key>
<value> <string></string> </value>
</item>
......@@ -20,7 +24,7 @@
</item>
<item>
<key> <string>after_script_name</string> </key>
<value> <string>Delivery_createRule</string> </value>
<value> <string></string> </value>
</item>
<item>
<key> <string>description</string> </key>
......
......@@ -56,7 +56,7 @@ if portal.portal_workflow.isTransitionPossible(context, \'calculate\'):\n
else:\n
# Make sure no other node is moving the delivery\n
# to \'diverged\' or \'solved\' state.\n
context.serialize()\n
context.serializeCausalityState()\n
</string> </value>
</item>
<item>
......
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="PythonScript" module="Products.PythonScripts.PythonScript"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>Script_magic</string> </key>
<value> <int>3</int> </value>
</item>
<item>
<key> <string>_bind_names</string> </key>
<value>
<object>
<klass>
<global name="NameAssignments" module="Shared.DC.Scripts.Bindings"/>
</klass>
<tuple/>
<state>
<dictionary>
<item>
<key> <string>_asgns</string> </key>
<value>
<dictionary>
<item>
<key> <string>name_container</string> </key>
<value> <string>container</string> </value>
</item>
<item>
<key> <string>name_context</string> </key>
<value> <string>context</string> </value>
</item>
<item>
<key> <string>name_m_self</string> </key>
<value> <string>script</string> </value>
</item>
<item>
<key> <string>name_subpath</string> </key>
<value> <string>traverse_subpath</string> </value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</state>
</object>
</value>
</item>
<item>
<key> <string>_body</string> </key>
<value> <string>from Products.ERP5Type.Errors import SimulationError\n
\n
delivery = context\n
delivery_type = context.getPortalType()\n
\n
# XXX The following dict is only for backward compatibility.\n
applied_rule_dict = {\n
\'Pay Sheet Transaction\': \'default_invoice_rule\',\n
\'Payment Transaction\': \'default_invoice_rule\',\n
\n
\'Purchase Packing List\': \'default_delivery_rule\',\n
\'Purchase Invoice Transaction\': \'default_invoice_rule\',\n
\n
\'Sale Order\': \'default_order_rule\',\n
\'Sale Packing List\': \'default_delivery_rule\',\n
\'Sale Invoice Transaction\': \'default_invoice_rule\',\n
\n
\'Internal Packing List\': None,\n
\'Returned Sale Packing List\': None,\n
\n
\'Accounting Rule Cell\': None,\n
\'Accounting Transaction\': None,\n
\'Production Packing List\': None,\n
\'Production Report\': None,\n
\'Pay Sheet Model\': None,\n
\n
\'Amortisation Transaction\' : None,\n
\'Task Report\': \'default_delivery_rule\',\n
}\n
\n
try:\n
applied_rule = delivery.getRuleReference()\n
except SimulationError:\n
marker = []\n
applied_rule = applied_rule_dict.get(delivery_type, marker)\n
if applied_rule is marker:\n
raise\n
\n
if applied_rule is None:\n
# No need to add a rule, but still we need to expand the delivery\n
# if at least one of its movements is simulated.\n
for m in delivery.getMovementList():\n
if m.isSimulated():\n
delivery.activate(activate_kw=activate_kw).expand(**kw)\n
return\n
elif applied_rule:\n
delivery.updateAppliedRule(rule_reference=applied_rule, activate_kw=activate_kw, **kw)\n
</string> </value>
</item>
<item>
<key> <string>_params</string> </key>
<value> <string>activate_kw=None, **kw</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>Delivery_updateAppliedRule</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="PythonScript" module="Products.PythonScripts.PythonScript"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>Script_magic</string> </key>
<value> <int>3</int> </value>
</item>
<item>
<key> <string>_bind_names</string> </key>
<value>
<object>
<klass>
<global name="NameAssignments" module="Shared.DC.Scripts.Bindings"/>
</klass>
<tuple/>
<state>
<dictionary>
<item>
<key> <string>_asgns</string> </key>
<value>
<dictionary>
<item>
<key> <string>name_container</string> </key>
<value> <string>container</string> </value>
</item>
<item>
<key> <string>name_context</string> </key>
<value> <string>context</string> </value>
</item>
<item>
<key> <string>name_m_self</string> </key>
<value> <string>script</string> </value>
</item>
<item>
<key> <string>name_subpath</string> </key>
<value> <string>traverse_subpath</string> </value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</state>
</object>
</value>
</item>
<item>
<key> <string>_body</string> </key>
<value> <string>delivery = context\n
delivery_path = delivery.getPath()\n
expand_tag = delivery_path + \'_expand\'\n
tag = delivery_path + \'_updateAppliedRule\'\n
\n
priority = 3\n
\n
# These parameters are passed to activate for expand and reindexObject,\n
# so Delivery_updateAppliedRule will wait for the creation and indexing of\n
# Applied Rules and Simulation Movements by another Delivery_updateAppliedRule.\n
# This is required for finding an existing Applied Rule by the catalog, and\n
# avoiding needless conflicts.\n
activate_kw = { \n
\'tag\': expand_tag,\n
\'priority\': priority,\n
}\n
\n
# Serialization is required for avoiding parallel executions of updateAppliedRule\n
# which may generate multiple Root Applied Rules.\n
delivery.activate(\n
after_tag=expand_tag,\n
tag=tag,\n
priority=priority,\n
serialization_tag=tag,\n
).Delivery_updateAppliedRule(activate_kw=activate_kw)\n
</string> </value>
</item>
<item>
<key> <string>_params</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>Delivery_updateSimulation</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
......@@ -28,7 +28,7 @@
<key> <string>after_script_name</string> </key>
<value>
<list>
<string>Delivery_calculateCausalityState</string>
<string>Delivery_calculate</string>
</list>
</value>
</item>
......
......@@ -59,7 +59,7 @@
</item>
<item>
<key> <string>id</string> </key>
<value> <string>Delivery_calculateCausalityState</string> </value>
<value> <string>Delivery_calculate</string> </value>
</item>
</dictionary>
</pickle>
......
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="PythonScript" module="Products.PythonScripts.PythonScript"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>Script_magic</string> </key>
<value> <int>3</int> </value>
</item>
<item>
<key> <string>_bind_names</string> </key>
<value>
<object>
<klass>
<global name="NameAssignments" module="Shared.DC.Scripts.Bindings"/>
</klass>
<tuple/>
<state>
<dictionary>
<item>
<key> <string>_asgns</string> </key>
<value>
<dictionary>
<item>
<key> <string>name_container</string> </key>
<value> <string>container</string> </value>
</item>
<item>
<key> <string>name_context</string> </key>
<value> <string>context</string> </value>
</item>
<item>
<key> <string>name_m_self</string> </key>
<value> <string>script</string> </value>
</item>
<item>
<key> <string>name_subpath</string> </key>
<value> <string>traverse_subpath</string> </value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</state>
</object>
</value>
</item>
<item>
<key> <string>_body</string> </key>
<value> <string>delivery = state_change[\'object\']\n
activate_kw = state_change[\'kwargs\'].get(\'activate_kw\') or {}\n
tag = delivery.getPath() + \'_calculate\'\n
delivery.activate(after_tag=tag, **activate_kw).updateCausalityState()\n
</string> </value>
</item>
<item>
<key> <string>_params</string> </key>
<value> <string>state_change</string> </value>
</item>
<item>
<key> <string>_proxy_roles</string> </key>
<value>
<tuple>
<string>Manager</string>
</tuple>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>Delivery_afterEdit</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
......@@ -60,8 +60,6 @@ split_and_defer = 0\n
listbox = state_change[\'kwargs\'].get(\'listbox\')\n
split_movement_list = []\n
if listbox is not None:\n
# Create Delivery Applied Rule (if required)\n
delivery.Delivery_updateAppliedRule()\n
for line in listbox:\n