See 32558. Running this sub in parallel within multiple transactions creates a deadlock. C4::ImportBatch::_update_batch_record_counts(): DBI Exception: DBD::mysql::st execute failed: Deadlock found when trying to get lock; try restarting transaction at /usr/share/koha/C4/ImportBatch.pm line 392
Created attachment 146752 [details] [review] Bug 32990: Prevent deadlock in _update_batch_record_counts Resolves: C4::ImportBatch::_update_batch_record_counts(): DBI Exception: DBD::mysql::st execute failed: Deadlock found when trying to get lock; try restarting transaction at /usr/share/koha/C4/ImportBatch.pm line 392 See also bug 32558. Test plan: If you apply 32558 first, run multiple processes that stage a marc import. Without this patch, you can trigger the deadlock. With this patch, it works. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Hi Marcel, I tried to test this, but I haven't been able to reproduce the deadlock. Were you stopping the worker daemons, queueing up multiple MARC imports, and then manually restarting the worker as in the test plan for bug 32558? Anything else to keep in mind to make the deadlock more likely?
(In reply to Emily Lamancusa from comment #2) > Hi Marcel, I tried to test this, but I haven't been able to reproduce the > deadlock. Were you stopping the worker daemons, queueing up multiple MARC > imports, and then manually restarting the worker as in the test plan for bug > 32558? Anything else to keep in mind to make the deadlock more likely? Hi Emily. Thx for trying. Yes, I stopped and restarted later. My test plan was a bit too compact, i am afraid..
Created attachment 147785 [details] [review] Bug 32990: Prevent deadlock in _update_batch_record_counts Resolves: C4::ImportBatch::_update_batch_record_counts(): DBI Exception: DBD::mysql::st execute failed: Deadlock found when trying to get lock; try restarting transaction at /usr/share/koha/C4/ImportBatch.pm line 392 See also bug 32558. Test plan: If you apply 32558 first, run multiple processes that stage a marc import. Without this patch, you can trigger the deadlock. With this patch, it works. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This worked, however, the error I got was: C4::ImportBatch::SetImportRecordMatches(): DBI Exception: DBD::mysql::st execute failed: Deadlock found when trying to get lock; try restarting transaction at /kohadevbox/koha/C4/ImportBatch.pm line 494 Does the statement in that routine need an update too?
(In reply to Nick Clemens from comment #5) > This worked, however, the error I got was: > > C4::ImportBatch::SetImportRecordMatches(): DBI Exception: DBD::mysql::st > execute failed: Deadlock found when trying to get lock; try restarting > transaction at /kohadevbox/koha/C4/ImportBatch.pm line 494 > > > Does the statement in that routine need an update too? Looks like it doesnt. You just hit the same problem in another part of the code execution in backgroundjob->process. It first does Stage and later calls FindDup.
Created attachment 150369 [details] [review] Bug 32990: Prevent deadlock in _update_batch_record_counts Resolves: C4::ImportBatch::_update_batch_record_counts(): DBI Exception: DBD::mysql::st execute failed: Deadlock found when trying to get lock; try restarting transaction at /usr/share/koha/C4/ImportBatch.pm line 392 See also bug 32558. Test plan: If you apply 32558 first, run multiple processes that stage a marc import. Without this patch, you can trigger the deadlock. With this patch, it works. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Pushed to master for 23.05. Nice work everyone, thanks!
Many hands makes light work, thankyou everyone! Pushed to 22.11.x for the next release
We have reports about import stuck at 0% on the mailing list, also with 22.11.06-03 - could this be related in any way? We've also seen some strange behaviour in 22.11.05. https://lists.katipo.co.nz/pipermail/koha/2023-June/059555.html
Backported to 22.05.x for upcoming 22.05.14
Does this need to be backported to 21.11.x?
(In reply to Danyon Sewell from comment #12) > Does this need to be backported to 21.11.x? See the connection to bug 32558 (not in 21.11). I think it is not needed. But if it applies, should not harm either?