Bugzilla – Attachment 159413 Details for
Bug 35438
Importing records can create too large transactions
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Help
|
New Account
|
Log In
[x]
|
Forgot Password
Login:
[x]
[patch]
Bug 35438: Transact each record import separately
Bug-35438-Transact-each-record-import-separately.patch (text/plain), 2.34 KB, created by
Kyle M Hall (khall)
on 2023-11-30 17:11:58 UTC
(
hide
)
Description:
Bug 35438: Transact each record import separately
Filename:
MIME Type:
Creator:
Kyle M Hall (khall)
Created:
2023-11-30 17:11:58 UTC
Size:
2.34 KB
patch
obsolete
>From e7e399a97ca7e1c58645ba0e7a691bd907a30007 Mon Sep 17 00:00:00 2001 >From: Kyle M Hall <kyle@bywatersolutions.com> >Date: Thu, 30 Nov 2023 12:10:41 -0500 >Subject: [PATCH] Bug 35438: Transact each record import separately > >When importing a staged file we commit every 50 records >For an authority import we are also merging, which can affect many more biblios, and these all end up in the transaction. >This can cause tables locks and issues across Koha > >Test Plan: >1) Apply this patch >2) prove t/db_dependent/ImportBatch.t >--- > C4/ImportBatch.pm | 9 ++------- > 1 file changed, 2 insertions(+), 7 deletions(-) > >diff --git a/C4/ImportBatch.pm b/C4/ImportBatch.pm >index 00fd7f3d436..24a7d1d4d65 100644 >--- a/C4/ImportBatch.pm >+++ b/C4/ImportBatch.pm >@@ -538,9 +538,6 @@ sub BatchCommitRecords { > $progress_interval = 0 unless ref($progress_callback) eq 'CODE'; > > my $schema = Koha::Database->schema; >- $schema->txn_begin; >- # NOTE: Moved this transaction to the front of the routine. Note that inside the while loop below >- # transactions may be committed and started too again. The final commit is close to the end. > > my $record_type; > my $num_added = 0; >@@ -571,14 +568,13 @@ sub BatchCommitRecords { > my @biblio_ids; > my @updated_ids; > while (my $rowref = $sth->fetchrow_hashref) { >+ $schema->txn_begin; > $record_type = $rowref->{'record_type'}; > > $rec_num++; > > if ($progress_interval and (0 == ($rec_num % $progress_interval))) { > # report progress and commit >- $schema->txn_commit unless $skip_intermediate_commit; >- &$progress_callback( $rec_num ); > $schema->txn_begin unless $skip_intermediate_commit; > } > if ($rowref->{'status'} eq 'error' or $rowref->{'status'} eq 'imported') { >@@ -708,6 +704,7 @@ sub BatchCommitRecords { > } > SetImportRecordStatus($rowref->{'import_record_id'}, 'ignored'); > } >+ $schema->txn_commit; > } > > if ($progress_interval){ >@@ -718,8 +715,6 @@ sub BatchCommitRecords { > > SetImportBatchStatus($batch_id, 'imported'); > >- # final commit should be before Elastic background indexing in order to find job data >- $schema->txn_commit; > > if (@biblio_ids) { > my $indexer = Koha::SearchEngine::Indexer->new( { index => $Koha::SearchEngine::BIBLIOS_INDEX } ); >-- >2.30.2
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Diff
|
Splinter Review
Attachments on
bug 35438
:
159413
|
159415
|
159442
|
159443
|
160917
|
160918