Use REPLACE rather than INSERT for DELETE..INSERT indexation pattern.
When a document gets rows inserted in category table while there was none before (typically: first document indexation), it may trigger IntegrityError: (1062, "Duplicate entry '...' for key 'PRIMARY'") because in the DELETE..INSERT pattern, DELETE finds no matching rows and does not gap-lock (because we enable innodb_locks_unsafe_for_binlog), then the second INSERT happening, chronologically speaking, waits for the transaction of the first to e committed, and on commit it causes such duplicate key error. A transient visible effect of this change is that if both competing indexations see a different document state (because document got modified in some 3rd transaction), the union of the resulting set of rows will be visible until the reindexation which must have been triggered by the 3rd transaction gets executed, at which point only the latest set will be visible. A similar issue exists before this change, with stricter conditions: it needs the intersection of both sets to be empty, because a non-empty intersection causes the duplicate key error solved here. This change has been measured to improve scalability of an invoice building test case (naturally triggering indexations) starting from ~12 activity nodes: 8 nodes: +1.4% invoices/hour 12 nodes: +9.5% invoices/hour 16 nodes: +12.3% invoices/hour (values are the difference between DELETE..INSERT and DELETE..REPLACE)
Showing
Please register or sign in to comment