Commit 2a84df59 authored by Vincent Pelletier's avatar Vincent Pelletier

Improve "ORDER BY" condition:

- remove uid and date sort, they causes pathologic cases in
  "site reindexation" payloads:
  Module indexation creates per-module batches of activities. All activities
  in a given batch will have identical date values, so indexation will happen
  on only one module at a time, which creates a bottleneck on setups where
  modules are scattered to multiple ZEOs (precisely to paralellise work on
  different modules).
- random sort added for multinode setups (see dtml comment, this is serious)


git-svn-id: https://svn.erp5.org/repos/public/erp5/trunk@29672 20353a03-c40f-0410-a6d1-a30d3c3de9de
parent 364149d5
...@@ -21,7 +21,16 @@ WHERE ...@@ -21,7 +21,16 @@ WHERE
AND date <= <dtml-sqlvar to_date type="datetime"> AND date <= <dtml-sqlvar to_date type="datetime">
<dtml-if expr="group_method_id is not None"> AND group_method_id = <dtml-sqlvar group_method_id type="string"> </dtml-if> <dtml-if expr="group_method_id is not None"> AND group_method_id = <dtml-sqlvar group_method_id type="string"> </dtml-if>
ORDER BY ORDER BY
priority, date, uid <dtml-comment>
Explanation of the order by:
- priority must be respected (it is a feature)
- when multiple nodes simultaneously try to fetch activities, they should not
be given the same set of lines as it would cause all minus one to wait for
a write lock (and be ultimately aborted), effectively serializing their
action (so breaking paralellism).
So we must force MySQL to update lines in a random order.
</dtml-comment>
priority, RAND()
LIMIT <dtml-sqlvar count type="int"> LIMIT <dtml-sqlvar count type="int">
<dtml-var sql_delimiter> <dtml-var sql_delimiter>
COMMIT COMMIT
...@@ -19,7 +19,16 @@ WHERE ...@@ -19,7 +19,16 @@ WHERE
processing_node=0 processing_node=0
AND date <= <dtml-sqlvar to_date type="datetime"> AND date <= <dtml-sqlvar to_date type="datetime">
ORDER BY ORDER BY
priority, date, uid <dtml-comment>
Explanation of the order by:
- priority must be respected (it is a feature)
- when multiple nodes simultaneously try to fetch activities, they should not
be given the same set of lines as it would cause all minus one to wait for
a write lock (and be ultimately aborted), effectively serializing their
action (so breaking paralellism).
So we must force MySQL to update lines in a random order.
</dtml-comment>
priority, RAND()
LIMIT <dtml-sqlvar count type="int"> LIMIT <dtml-sqlvar count type="int">
<dtml-var sql_delimiter> <dtml-var sql_delimiter>
COMMIT COMMIT
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment