Commit 093c6e7a authored by Julien Muchembled's avatar Julien Muchembled Committed by Łukasz Nowak

CMFActivity: try to reserve older messages first (ORDER BY priority, date)

This reverts commit 2a84df59 partially.
With a good index, requests should be fast enough to avoid locks.

Use following requests to update activity tables:

ALTER TABLE message DROP KEY processing_node_date, ADD KEY processing_node_priority_date (processing_node, priority, date);
ALTER TABLE message_queue DROP KEY processing_node_date, ADD KEY processing_node_priority_date (processing_node, priority, date);
parent e2890911
......@@ -25,15 +25,22 @@ WHERE
</dtml-if>
ORDER BY
<dtml-comment>
Explanation of the order by:
- priority must be respected (it is a feature)
- when multiple nodes simultaneously try to fetch activities, they should not
be given the same set of lines as it would cause all minus one to wait for
a write lock (and be ultimately aborted), effectively serializing their
action (so breaking paralellism).
So we must force MySQL to update lines in a random order.
During normal operation, sorting by date (as 2nd criteria) is fairer
for users and reduce the probability to do the same work several times
(think of an object that is modified several times in a short period of time).
However, current implementation is not optimal when reindexing a whole site
with several mount points (to different ZEO servers), because modules may not
be processed in parallel. If you want to speed up ERP5Site_reindexAll,
consider:
- ordering by 'priority, RAND()' temporarily;
- or better, hack ERP5Site_reindexAll so that all reindex messages have
identical/random dates (hint: add optional parameter to Folder_reindexAll
and Folder_reindexObjectList in order to forward a date from
ERP5Site_reindexAll, e.g. current date would work if MySQL
shuffles enough lines with same priority/date).
- or even better, use NEO <http://www.neoppod.org/>
</dtml-comment>
priority, RAND()
priority, date
LIMIT <dtml-sqlvar count type="int">
<dtml-var sql_delimiter>
COMMIT
......@@ -29,7 +29,7 @@ CREATE TABLE `message` (
KEY (`active_process_uid`),
KEY (`method_id`),
KEY `processing_node_processing` (`processing_node`, `processing`),
KEY `processing_node_date` (`processing_node`, `date`),
KEY `processing_node_priority_date` (`processing_node`, `priority`, `date`),
KEY `serialization_tag_processing_node` (`serialization_tag`, `processing_node`),
KEY (`priority`),
KEY (`tag`),
......
......@@ -28,7 +28,7 @@ CREATE TABLE `message_queue` (
KEY (`active_process_uid`),
KEY (`method_id`),
KEY `processing_node_processing` (`processing_node`, `processing`),
KEY `processing_node_date` (`processing_node`, `date`),
KEY `processing_node_priority_date` (`processing_node`, `priority`, `date`),
KEY `serialization_tag_processing_node` (`serialization_tag`, `processing_node`),
KEY (`priority`),
KEY (`tag`)
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment