Bugzilla – Attachment 181997 Details for
Bug 37020
bulkmarcimport gets killed after update to 24.05 when inserting large files
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Help
|
New Account
|
Log In
[x]
|
Forgot Password
Login:
[x]
[patch]
Bug 37020: (Followup) Fix txn handling, loop label and record_number counter
Bug-37020-Followup-Fix-txn-handling-loop-label-and.patch (text/plain), 2.16 KB, created by
Thomas Klausner
on 2025-05-06 20:38:09 UTC
(
hide
)
Description:
Bug 37020: (Followup) Fix txn handling, loop label and record_number counter
Filename:
MIME Type:
Creator:
Thomas Klausner
Created:
2025-05-06 20:38:09 UTC
Size:
2.16 KB
patch
obsolete
>From 18049f2f87ee826790836f78397ddd13dc7933c3 Mon Sep 17 00:00:00 2001 >From: Thomas Klausner <domm@plix.at> >Date: Mon, 14 Apr 2025 13:17:11 +0000 >Subject: [PATCH] Bug 37020: (Followup) Fix txn handling, loop label and > record_number counter > >So, I reviewed the code, which has a bunch of issues: > >* there's a call to `next RECORD`, but no `RECORD` loop label. >* $record_number is incremented twice >* and the calls to txn_begin is moved from the outside the loop into each record, starting one transaction per record. Not sure how mysql handles this, but postgres would not like it. > >This patch fixes those issues: > >After I fixed the first two issues, I ran the patched script on a file containing 75 authorities (from GND), and not one was added to the DB, presuambly because nested transactions don't work in mysql? > >So I also moved the BEGIN transaction outside of the main loop and started a new one after each commit, and added a final commit after to loop. > >Signed-off-by: Jan Kissig <bibliothek@th-wildau.de> >Signed-off-by: Thomas Klausner <domm@plix.at> >--- > misc/migration_tools/bulkmarcimport.pl | 9 +++++++-- > 1 file changed, 7 insertions(+), 2 deletions(-) > >diff --git a/misc/migration_tools/bulkmarcimport.pl b/misc/migration_tools/bulkmarcimport.pl >index 638753e589..ca4f8b2bd3 100755 >--- a/misc/migration_tools/bulkmarcimport.pl >+++ b/misc/migration_tools/bulkmarcimport.pl >@@ -325,7 +325,8 @@ my $logger = Koha::Logger->get; > my $schema = Koha::Database->schema; > my $lint = MARC::Lint->new; > >-while () { >+$schema->txn_begin; >+RECORD: while () { > > my $record; > >@@ -475,7 +476,6 @@ while () { > } > } > unless ($test_parameter) { >- $schema->txn_begin; > > if ($authorities) { > my $authtypecode = GuessAuthTypeCode( $record, $heading_fields ); >@@ -726,10 +726,15 @@ while () { > @search_engine_records = (); > } > $schema->txn_commit; >+ $schema->txn_begin; > } > last if $record_number == $number || $records_exhausted; > } > >+if ( !$test_parameter ) { >+ $schema->txn_commit; >+} >+ > if ($fk_off) { > $dbh->do("SET FOREIGN_KEY_CHECKS = 1"); > } >-- >2.39.5
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Diff
|
Splinter Review
Attachments on
bug 37020
:
176388
|
176425
|
176653
|
176921
|
180639
|
180640
|
180641
|
180644
|
180645
|
180905
|
180906
|
181463
|
181470
|
181471
|
181472
|
181996
| 181997 |
181998