Bugzilla – Attachment 95414 Details for
Bug 24027
Adding multiple items is slow
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Help
|
New Account
|
Log In
[x]
|
Forgot Password
Login:
[x]
[patch]
Bug 24027: (follow-up) ModZebra should be called once after all items added, not on each item add
Bug-24027-follow-up-ModZebra-should-be-called-once.patch (text/plain), 5.68 KB, created by
Andrii Nugged
on 2019-11-14 00:02:05 UTC
(
hide
)
Description:
Bug 24027: (follow-up) ModZebra should be called once after all items added, not on each item add
Filename:
MIME Type:
Creator:
Andrii Nugged
Created:
2019-11-14 00:02:05 UTC
Size:
5.68 KB
patch
obsolete
>From 1b73726ba2b90b2152dd1b808d9fceb32ea5b6f6 Mon Sep 17 00:00:00 2001 >From: Andrew Nugged <nugged@gmail.com> >Date: Thu, 14 Nov 2019 01:54:56 +0200 >Subject: [PATCH] Bug 24027: (follow-up) ModZebra should be called once after > all items added, not on each item add > >Why happened: >- call to ModZebra was done after EACH item added, > but it was called only with main biblionumber only, > so call was the same on each of requests >- and also time spent in that ModZebra sub increased with every > next hundred items in DB for that element: so adding every next 100 > was slower and slower, > >Solved: >- now it's called only once (by adding some extra parameter to "AddItem*" sub set in postponed mode) >- and now adding of elements not so heavily depends from how much items was in DB before. > >Test plan / How to replicate the issue: >(test depends from how already populated DB and Zebra/Elasticsearch DBs > so might be not so noticeably fast on empty DBs) > >- go to one of bibliotitems cataloguing/additem.pl?biblionumber=XXX pages >- press button "add multiple copies of this item", put 1000 on slower machines, 5000 on faster >- start measuring time + submit page/form > >It takes some serious amount of time even up to timeout. >After applying patch, same procedure for same number of items. >Also it much less slows down when number of items rises, more linearly. > >But, again, it heavily depends how database is populated and indexes are rebuild >but it's obvious that it will me much more efficient call "ModZebra" once after 1000 addings, >then 1000 times on each item copied. >--- > C4/Items.pm | 17 ++++++++++++----- > cataloguing/additem.pl | 5 ++++- > 2 files changed, 16 insertions(+), 6 deletions(-) > >diff --git a/C4/Items.pm b/C4/Items.pm >index eb46859c24..3c1c0480dc 100644 >--- a/C4/Items.pm >+++ b/C4/Items.pm >@@ -144,7 +144,7 @@ sub CartToShelf { > =head2 AddItemFromMarc > > my ($biblionumber, $biblioitemnumber, $itemnumber) >- = AddItemFromMarc($source_item_marc, $biblionumber[, $dbh]); >+ = AddItemFromMarc($source_item_marc, $biblionumber[, $dbh, $postpone_indexes_update]); > > Given a MARC::Record object containing an embedded item > record and a biblionumber, create a new item record. >@@ -155,6 +155,7 @@ sub AddItemFromMarc { > my $source_item_marc = shift; > my $biblionumber = shift; > my $dbh = @_ ? shift : C4::Context->dbh; >+ my $postpone_indexes_update = @_ ? shift : undef; > > # parse item hash from MARC > my $frameworkcode = C4::Biblio::GetFrameworkCode($biblionumber); >@@ -164,13 +165,13 @@ sub AddItemFromMarc { > $localitemmarc->append_fields( $source_item_marc->field($itemtag) ); > my $item = C4::Biblio::TransformMarcToKoha( $localitemmarc, $frameworkcode, 'items' ); > my $unlinked_item_subfields = _get_unlinked_item_subfields( $localitemmarc, $frameworkcode ); >- return AddItem( $item, $biblionumber, $dbh, $frameworkcode, $unlinked_item_subfields ); >+ return AddItem( $item, $biblionumber, $dbh, $frameworkcode, $unlinked_item_subfields, $postpone_indexes_update ); > } > > =head2 AddItem > > my ($biblionumber, $biblioitemnumber, $itemnumber) >- = AddItem($item, $biblionumber[, $dbh, $frameworkcode, $unlinked_item_subfields]); >+ = AddItem($item, $biblionumber[, $dbh, $frameworkcode, $unlinked_item_subfields, $postpone_indexes_update]); > > Given a hash containing item column names as keys, > create a new Koha item record. >@@ -179,13 +180,17 @@ The first two optional parameters (C<$dbh> and C<$frameworkcode>) > do not need to be supplied for general use; they exist > simply to allow them to be picked up from AddItemFromMarc. > >-The final optional parameter, C<$unlinked_item_subfields>, contains >+Optional parameter, C<$unlinked_item_subfields>, contains > an arrayref containing subfields present in the original MARC > representation of the item (e.g., from the item editor) that are > not mapped to C<items> columns directly but should instead > be stored in C<items.more_subfields_xml> and included in > the biblio items tag for display and indexing. > >+The final optional parameter, C<$postpone_indexes_update>, prevents >+calling of ModZebra sub so this used when adding items in a batch and >+then ModZebra called in C<additem.pl> after the whole loop. >+ > =cut > > sub AddItem { >@@ -198,6 +203,7 @@ sub AddItem { > if (@_) { > $unlinked_item_subfields = shift; > } >+ my $postpone_indexes_update = @_ ? shift : undef; > > # needs old biblionumber and biblioitemnumber > $item->{'biblionumber'} = $biblionumber; >@@ -221,7 +227,8 @@ sub AddItem { > > $item->{'itemnumber'} = $itemnumber; > >- C4::Biblio::ModZebra( $item->{biblionumber}, "specialUpdate", "biblioserver" ); >+ C4::Biblio::ModZebra( $item->{biblionumber}, "specialUpdate", "biblioserver" ) >+ unless $postpone_indexes_update; > > logaction( "CATALOGUING", "ADD", $itemnumber, "item" ) > if C4::Context->preference("CataloguingLog"); >diff --git a/cataloguing/additem.pl b/cataloguing/additem.pl >index eded25fc5f..ca987a41b0 100755 >--- a/cataloguing/additem.pl >+++ b/cataloguing/additem.pl >@@ -600,7 +600,7 @@ if ($op eq "additem") { > > # Adding the item > if (!$exist_itemnumber) { >- my ($oldbiblionumber,$oldbibnum,$oldbibitemnum) = AddItemFromMarc( $record, $biblionumber, $dbh); >+ my ($oldbiblionumber,$oldbibnum,$oldbibitemnum) = AddItemFromMarc( $record, $biblionumber, $dbh, 1 ); > set_item_default_location($oldbibitemnum); > > # We count the item only if it was really added >@@ -614,6 +614,9 @@ if ($op eq "additem") { > # Preparing the next iteration > $oldbarcode = $barcodevalue; > } >+ >+ C4::Biblio::ModZebra( $biblionumber, "specialUpdate", "biblioserver" ); >+ > undef($itemrecord); > } > } >-- >2.17.1
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Diff
|
Splinter Review
Attachments on
bug 24027
:
95413
|
95414
|
95417
|
95446
|
95473
|
95663
|
98250
|
98251
|
98252
|
98253
|
102464
|
102474
|
102567
|
102568
|
102572
|
102578
|
102579
|
102580