Bugzilla – Attachment 187703 Details for
Bug 40986
Add some reindexing examples to rebuild_elasticsearch.pl
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Help
|
New Account
|
Log In
[x]
|
Forgot Password
Login:
[x]
[patch]
Bug 40986: add examples to rebuild_elasticsearch.pl --man
Bug-40986-add-examples-to-rebuildelasticsearchpl--.patch (text/plain), 7.79 KB, created by
Thomas Klausner
on 2025-10-09 18:47:48 UTC
(
hide
)
Description:
Bug 40986: add examples to rebuild_elasticsearch.pl --man
Filename:
MIME Type:
Creator:
Thomas Klausner
Created:
2025-10-09 18:47:48 UTC
Size:
7.79 KB
patch
obsolete
>From 38cfaf34e747baa2fd90d76d89cd84306f4f4eab Mon Sep 17 00:00:00 2001 >From: Thomas Klausner <domm@plix.at> >Date: Thu, 9 Oct 2025 18:08:12 +0000 >Subject: [PATCH] Bug 40986: add examples to rebuild_elasticsearch.pl --man > >This commit adds a bunch of example to rebuild_elasticsearch.pl --man, mostly about passing custom sql via --where. > >It also fixes a few trivial issues in rebuild_elasticsearch.pl and adds a note to debian/scripts/koha-elasticsearch to point people to the extended docs > >Test plan(s): > >1: No examples in detailed help > >* run: misc/search_tools/rebuild_elasticsearch.pl --man >* After OPTIONS (scroll down a bit with the down-arrow or hit space) the section IMPLEMENTATION is shown. >* No EXAMPLES section > >2: Ugly error message: > >* run: misc/search_tools/rebuild_elasticsearch.pl --processes 2 --authid 123 >* output: "Argument p|processes cannot be combined with bn|bnumber or ai|authid at misc/search_tools/rebuild_elasticsearch.pl line 163." >* Notice "at misc/search_tools/rebuild_elasticsearch.pl line 163" > >3: Warning: > >* run: misc/search_tools/rebuild_elasticsearch.pl -v --biblios --where 'biblionumber = -1' >* output: >[9574] Checking state of biblios index >[9574] Indexing biblios >[9574] Committing final records... >Use of uninitialized value in string eq at misc/search_tools/rebuild_elasticsearch.pl line 365. >[9574] Total 0 records indexed > >* Notice the warning "Use of uninitialized value.." > >4: Error when using --where without -b or -a > >* run: misc/search_tools/rebuild_elasticsearch.pl -v --where 'biblionumber = 1' >* output: >misc/search_tools/rebuild_elasticsearch.pl -v --where 'biblionumber = 420' >... >DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::mysql::st execute failed: Unknown column 'biblionumber' in 'where clause' at /kohadevbox/koha/Koha/MetadataRecord/Authority.pm line 239 > >* The script dies, because auth_header has not column biblionumber and we where running --where on both tables. > >5: No hint about reindex_elasticsearch in koha-elasticsearch > >* run: debian/scripts/koha-elasticsearch --help >* There is no hint about reindex_elasticsearch.pl > >Now apply the patch. No need to restart anything > >1: examples are shown in detailed help > >* run: misc/search_tools/rebuild_elasticsearch.pl --man >* after OPTIONS you will find a EXAMPLES section listing various use cases > >2: Ugly error message is not ugly anymore > >* run: misc/search_tools/rebuild_elasticsearch.pl --processes 2 --authid 123 >* output: Argument p|processes cannot be combined with bn|bnumber or ai|authid >* no more "at blabla line ..." > >3: Warning: > >* run: misc/search_tools/rebuild_elasticsearch.pl -v --biblios --where 'biblionumber = -1' >* output: >[9574] Checking state of biblios index >[9574] Indexing biblios >[9574] Committing final records... >[9574] Total 0 records indexed > >* Notice the warning "Use of uninitialized value.." is gone > >4: Error when using --where without -b or -a > >* run: misc/search_tools/rebuild_elasticsearch.pl -v --where 'biblionumber = 1' >* output: Argument w|where must be combined with either b|biblios or a|authorities >* and the script aborts. > >5: koha-elasticsearch mentions reindex_elasticsearch.pl > >* run: debian/scripts/koha-elasticsearch --help >* There **is** a hint about reindex_elasticsearch.pl > >Now apply the patch. No need to restart anything > >Sponsored-by: HKS3 >Sponsored-by: Koha DACH Hackfest >--- > debian/scripts/koha-elasticsearch | 3 + > misc/search_tools/rebuild_elasticsearch.pl | 74 +++++++++++++++++++++- > 2 files changed, 75 insertions(+), 2 deletions(-) > >diff --git a/debian/scripts/koha-elasticsearch b/debian/scripts/koha-elasticsearch >index 851e97f49b..63325b9e80 100755 >--- a/debian/scripts/koha-elasticsearch >+++ b/debian/scripts/koha-elasticsearch >@@ -49,6 +49,9 @@ Options: > --verbose|-v Be verbose. > --help|-h Print this help. > >+This is a wrapper script for rebuild_elasticsearch.pl, which contains more >+documentation and might have more options. >+ > EOF > } > >diff --git a/misc/search_tools/rebuild_elasticsearch.pl b/misc/search_tools/rebuild_elasticsearch.pl >index eb10a1b378..0481e7252c 100755 >--- a/misc/search_tools/rebuild_elasticsearch.pl >+++ b/misc/search_tools/rebuild_elasticsearch.pl >@@ -112,6 +112,73 @@ Full documentation. > > =back > >+=head1 EXAMPLES >+ >+C<rebuild_elasticsearch.pl> >+ >+=head2 Index a single biblio or authority >+ >+C<rebuild_elasticsearch.pl -v -bn 123> >+ >+C<rebuild_elasticsearch.pl -v -ai 456> >+ >+This is useful when playing with new mappings. >+ >+=head2 Use C<--where> to select the records to index >+ >+You can pass arbitrary SQL via C<--where> to select which records you >+want to index. C<--where> only works if you specify either C<-b> or >+C<-a> to index either biblios or authorities. >+ >+When using C<-b> to index biblios, the SQL is applied to the table >+C<biblio>. When indexing authorities via C<-a>, the table >+C<auth_header> is used. >+ >+Please note that you cannot do proper joins (because this would make >+the CLI very complicated). But you can use subselects. >+ >+=head3 biblios created in the last 24h >+ >+ rebuild_elasticsearch.pl -v --biblios \ >+ --where 'datecreated > date_sub(now(), interval 1 day)' >+ >+=head3 authorities with authid between 500 and 600 >+ >+ rebuild_elasticsearch.pl -v --authorities \ >+ --where 'authid between 500 and 599' >+ >+=head3 biblios with an item that was modified in the last hour OR metadata was modified in the last hour >+ >+ rebuild_elasticsearch.pl -v --biblios \ >+ --where " biblionumber in ( select biblionumber from biblio_metadata where timestamp >= date_sub(now(), interval 1 hour)) >+ OR biblionumber in (select biblionumber from items where timestamp >= date_sub(now(), interval 1 hour)) " >+ >+Or use a UNION: >+ >+ rebuild_elasticsearch.pl -v --biblios \ >+ --where "biblionumber in ( >+ select biblionumber from biblio_metadata where timestamp >= '2025-10-09' >+ UNION select biblionumber from items where timestamp >= '2025-10-09') " >+ >+=head3 biblios where 856u starts with 'http' >+ >+ rebuild_elasticsearch.pl -v --biblios \ >+ --where 'biblionumber in ( >+ select biblionumber from biblio_metadata where >+ ExtractValue(metadata,'\''//datafield[@tag="856"]/subfield[@code="u"]'\'') like "http%")' >+ >+In this case we have to be quite careful with the bash quoting. We use single quotes for C<--where> (so that the C<@> sign is not interpreted from bash) and the have to esacpe the single quotes we need in C<ExtractValue> with C<'\''>. >+ >+=head3 Very complex queries >+ >+For very complex queries it is easier to create a table in the database containing the biblionumbers and then use a subselect to select them. >+ >+ CREATE TABLE some_selection AS SELECT biblionumber FROM ... MONSTER SQL ...; >+ >+ rebuild_elasticsearch.pl -v --biblios --where ' biblionumber in (select biblionumber from some_selection ); >+ >+ DROP TABLE some_selection; >+ > =head1 IMPLEMENTATION > > =cut >@@ -160,7 +227,7 @@ unless ( $index_authorities || $index_biblios ) { > } > > if ( $processes && ( @biblionumbers || @authids ) ) { >- die "Argument p|processes cannot be combined with bn|bnumber or ai|authid"; >+ die "Argument p|processes cannot be combined with bn|bnumber or ai|authid\n"; > } > > pod2usage(1) if $help; >@@ -207,6 +274,9 @@ if ($desc) { > } > > if ($where) { >+ if ( $index_authorities && $index_biblios ) { >+ die "Argument w|where must be combined with either b|biblios or a|authorities\n"; >+ } > $iterator_options{where} = $where; > } > >@@ -362,7 +432,7 @@ Parse the return from update_index and display errors depending on verbosity of > > sub _handle_response { > my ($response) = @_; >- if ( $response->{errors} eq 'true' ) { >+ if ( $response && $response->{errors} eq 'true' ) { > _log( 1, "There were errors during indexing\n" ); > if ( $verbose > 1 ) { > foreach my $item ( @{ $response->{items} } ) { >-- >2.39.5
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Diff
|
Splinter Review
Attachments on
bug 40986
: 187703