Commit 093c6e7a authored by Julien Muchembled's avatar Julien Muchembled Committed by Łukasz Nowak

CMFActivity: try to reserve older messages first (ORDER BY priority, date)

This reverts commit 2a84df59 partially.
With a good index, requests should be fast enough to avoid locks.

Use following requests to update activity tables:

ALTER TABLE message DROP KEY processing_node_date, ADD KEY processing_node_priority_date (processing_node, priority, date);
ALTER TABLE message_queue DROP KEY processing_node_date, ADD KEY processing_node_priority_date (processing_node, priority, date);
parent e2890911
...@@ -25,15 +25,22 @@ WHERE ...@@ -25,15 +25,22 @@ WHERE
</dtml-if> </dtml-if>
ORDER BY ORDER BY
<dtml-comment> <dtml-comment>
Explanation of the order by: During normal operation, sorting by date (as 2nd criteria) is fairer
- priority must be respected (it is a feature) for users and reduce the probability to do the same work several times
- when multiple nodes simultaneously try to fetch activities, they should not (think of an object that is modified several times in a short period of time).
be given the same set of lines as it would cause all minus one to wait for However, current implementation is not optimal when reindexing a whole site
a write lock (and be ultimately aborted), effectively serializing their with several mount points (to different ZEO servers), because modules may not
action (so breaking paralellism). be processed in parallel. If you want to speed up ERP5Site_reindexAll,
So we must force MySQL to update lines in a random order. consider:
- ordering by 'priority, RAND()' temporarily;
- or better, hack ERP5Site_reindexAll so that all reindex messages have
identical/random dates (hint: add optional parameter to Folder_reindexAll
and Folder_reindexObjectList in order to forward a date from
ERP5Site_reindexAll, e.g. current date would work if MySQL
shuffles enough lines with same priority/date).
- or even better, use NEO <http://www.neoppod.org/>
</dtml-comment> </dtml-comment>
priority, RAND() priority, date
LIMIT <dtml-sqlvar count type="int"> LIMIT <dtml-sqlvar count type="int">
<dtml-var sql_delimiter> <dtml-var sql_delimiter>
COMMIT COMMIT
...@@ -29,7 +29,7 @@ CREATE TABLE `message` ( ...@@ -29,7 +29,7 @@ CREATE TABLE `message` (
KEY (`active_process_uid`), KEY (`active_process_uid`),
KEY (`method_id`), KEY (`method_id`),
KEY `processing_node_processing` (`processing_node`, `processing`), KEY `processing_node_processing` (`processing_node`, `processing`),
KEY `processing_node_date` (`processing_node`, `date`), KEY `processing_node_priority_date` (`processing_node`, `priority`, `date`),
KEY `serialization_tag_processing_node` (`serialization_tag`, `processing_node`), KEY `serialization_tag_processing_node` (`serialization_tag`, `processing_node`),
KEY (`priority`), KEY (`priority`),
KEY (`tag`), KEY (`tag`),
......
...@@ -28,7 +28,7 @@ CREATE TABLE `message_queue` ( ...@@ -28,7 +28,7 @@ CREATE TABLE `message_queue` (
KEY (`active_process_uid`), KEY (`active_process_uid`),
KEY (`method_id`), KEY (`method_id`),
KEY `processing_node_processing` (`processing_node`, `processing`), KEY `processing_node_processing` (`processing_node`, `processing`),
KEY `processing_node_date` (`processing_node`, `date`), KEY `processing_node_priority_date` (`processing_node`, `priority`, `date`),
KEY `serialization_tag_processing_node` (`serialization_tag`, `processing_node`), KEY `serialization_tag_processing_node` (`serialization_tag`, `processing_node`),
KEY (`priority`), KEY (`priority`),
KEY (`tag`) KEY (`tag`)
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment