Bugzilla – Attachment 85976 Details for
Bug 21872
Elasticsearch indexing faster by making it multi-threaded
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Help
|
New Account
|
Log In
[x]
|
Forgot Password
Login:
[x]
[patch]
Bug 21872: Simplify conditions and exit on invalid combination of arguments
Bug-21872-Simplify-conditions-and-exit-on-invalid-.patch (text/plain), 5.96 KB, created by
Ere Maijala
on 2019-03-04 12:36:54 UTC
(
hide
)
Description:
Bug 21872: Simplify conditions and exit on invalid combination of arguments
Filename:
MIME Type:
Creator:
Ere Maijala
Created:
2019-03-04 12:36:54 UTC
Size:
5.96 KB
patch
obsolete
>From fb4b9a3dc55add248e22380c998017f7eaac6bbe Mon Sep 17 00:00:00 2001 >From: David Gustafsson <david.gustafsson@ub.gu.se> >Date: Thu, 14 Feb 2019 15:06:35 +0100 >Subject: [PATCH] Bug 21872: Simplify conditions and exit on invalid > combination of arguments > >Change to zero based indexing for slice index to simplify some >conditions. Exit with error message if trying to combine processes >and biblio numbers arguments. >--- > Koha/BiblioUtils.pm | 2 -- > Koha/MetadataRecord/Authority.pm | 2 -- > misc/search_tools/rebuild_elastic_search.pl | 23 +++++++++++---------- > 3 files changed, 12 insertions(+), 15 deletions(-) > >diff --git a/Koha/BiblioUtils.pm b/Koha/BiblioUtils.pm >index c83d94569d..88395a952f 100644 >--- a/Koha/BiblioUtils.pm >+++ b/Koha/BiblioUtils.pm >@@ -131,8 +131,6 @@ sub get_all_biblios_iterator { > if ($options{slice}) { > $slice_count = $options{slice}->{count}; > $slice_modulo = $options{slice}->{index}; >- $slice_modulo = 0 if ($slice_modulo == $slice_count); >- > $search_terms = \[ ' mod(biblionumber, ?) = ?', $slice_count, $slice_modulo]; > } > >diff --git a/Koha/MetadataRecord/Authority.pm b/Koha/MetadataRecord/Authority.pm >index faacd44ecc..48aa7aefa9 100644 >--- a/Koha/MetadataRecord/Authority.pm >+++ b/Koha/MetadataRecord/Authority.pm >@@ -182,8 +182,6 @@ sub get_all_authorities_iterator { > if ($options{slice}) { > $slice_count = $options{slice}->{count}; > $slice_modulo = $options{slice}->{index}; >- $slice_modulo = 0 if ($slice_modulo == $slice_count); >- > $search_terms->{authid} = \[ ' mod ? = ?', $slice_count, $slice_modulo]; > $search_terms = { > '-and' => [ >diff --git a/misc/search_tools/rebuild_elastic_search.pl b/misc/search_tools/rebuild_elastic_search.pl >index 07c313a9ec..8064c91330 100755 >--- a/misc/search_tools/rebuild_elastic_search.pl >+++ b/misc/search_tools/rebuild_elastic_search.pl >@@ -124,6 +124,10 @@ unless ($index_authorities || $index_biblios) { > $index_authorities = $index_biblios = 1; > } > >+if ($processes && @record_numbers) { >+ die "Argument p|processes cannot be combined with bn|bnumber"; >+} >+ > pod2usage(1) if $help; > pod2usage( -exitstatus => 0, -verbose => 2 ) if $man; > >@@ -134,11 +138,12 @@ _verify_index_state($Koha::SearchEngine::Elasticsearch::AUTHORITIES_INDEX, $dele > > my $slice_index = 0; > my $slice_count = $processes // 1; >+my %iterator_options; > > if ($slice_count > 1) { > # Fire up child processes for processing slices from 2 on. This main process will handle slice 1. >- $slice_index = 1; >- for (my $proc = 2; $proc <= $processes; $proc++) { >+ $slice_index = 0; >+ for (my $proc = 1; $proc < $slice_count; $proc++) { > my $pid = fork(); > die "Failed to fork a child process\n" unless defined $pid; > if ($pid == 0) { >@@ -148,12 +153,8 @@ if ($slice_count > 1) { > } > } > # Fudge the commit count a bit to spread out the Elasticsearch commits >- $commit *= 1 + 0.10 * ($slice_index - 1); >-} >- >-my %iterator_options; >-if ($slice_index) { >- _log(1, "Processing slice $slice_index of $slice_count\n"); >+ $commit *= 1 + 0.10 * $slice_index; >+ _log(1, "Processing slice @{[$slice_index + 1]} of $slice_count\n"); > $iterator_options{slice} = { index => $slice_index, count => $slice_count }; > } > >@@ -192,16 +193,16 @@ if ($index_authorities) { > _do_reindex($next, $Koha::SearchEngine::Elasticsearch::AUTHORITIES_INDEX); > } > >-if ($slice_index == 1) { >+if ($slice_index == 0) { > # Main process, wait for children >- for (my $proc = 2; $proc <= $processes; $proc++) { >+ for (my $proc = 1; $proc < $processes; $proc++) { > wait(); > } > } > > =head2 _verify_index_state > >- _ verify_index_state($Koha::SearchEngine::Elasticsearch::BIBLIOS_INDEX, 1); >+ _verify_index_state($Koha::SearchEngine::Elasticsearch::BIBLIOS_INDEX, 1); > > Checks the index state and recreates it if requested. > >-- >2.17.1
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Diff
|
Splinter Review
Attachments on
bug 21872
:
82582
|
82583
|
82584
|
82585
|
82586
|
82598
|
82599
|
82780
|
85119
|
85121
|
85975
|
85976
|
85977
|
85978
|
85979
|
86053
|
86054
|
86055
|
86056
|
86057
|
89109
|
89110
|
89111
|
89112
|
89113