Bug 26180 - Elasticsearch - Add option to index records in descending order
Summary: Elasticsearch - Add option to index records in descending order
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching - Elasticsearch (show other bugs)
Version: Main
Hardware: All All
: P5 - low enhancement
Assignee: Nick Clemens (kidclamp)
QA Contact:
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-08-07 13:14 UTC by Nick Clemens (kidclamp)
Modified: 2022-06-06 20:25 UTC (History)
3 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
20.11.00
Circulation function:


Attachments
Bug 26180: Add descending option to rebuild_elasticsearch.pl (4.40 KB, patch)
2020-08-07 13:21 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review
Bug 26180: Add descending option to rebuild_elasticsearch.pl (4.53 KB, patch)
2020-10-22 08:07 UTC, Séverine Queune
Details | Diff | Splinter Review
Bug 26180: Add descending option to rebuild_elasticsearch.pl (4.61 KB, patch)
2020-10-22 10:43 UTC, Martin Renvoize (ashimema)
Details | Diff | Splinter Review
Bug 26180: Add descending option to rebuild_elasticsearch.pl (4.86 KB, patch)
2020-11-04 08:53 UTC, Jonathan Druart
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Clemens (kidclamp) 2020-08-07 13:14:27 UTC
While the ES index is incremental and provides results as it commits, we currently index from the oldest records to the newest.

It would be nice to provide the option to go the other direction
Comment 1 Nick Clemens (kidclamp) 2020-08-07 13:21:07 UTC
Created attachment 107955 [details] [review]
Bug 26180: Add descending option to rebuild_elasticsearch.pl

TO test:
 1 - Have ES setup and running for Koha
 2 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b
 3 - Note the biblios index from number 1 the end
 4 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a
 5 - Notice the same
 6 - Apply patch
 7 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b
 8 - Still in ascending order
 9 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b --desc
10 - Now records index in descending order
11 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a
12 - Still ascending
13 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a --desc
14 - Now descending
Comment 2 Séverine Queune 2020-10-22 08:07:25 UTC
Created attachment 112146 [details] [review]
Bug 26180: Add descending option to rebuild_elasticsearch.pl

TO test:
 1 - Have ES setup and running for Koha
 2 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b
 3 - Note the biblios index from number 1 the end
 4 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a
 5 - Notice the same
 6 - Apply patch
 7 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b
 8 - Still in ascending order
 9 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b --desc
10 - Now records index in descending order
11 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a
12 - Still ascending
13 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a --desc
14 - Now descending

Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>
Comment 3 Martin Renvoize (ashimema) 2020-10-22 10:43:41 UTC
Created attachment 112159 [details] [review]
Bug 26180: Add descending option to rebuild_elasticsearch.pl

TO test:
 1 - Have ES setup and running for Koha
 2 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b
 3 - Note the biblios index from number 1 the end
 4 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a
 5 - Notice the same
 6 - Apply patch
 7 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b
 8 - Still in ascending order
 9 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b --desc
10 - Now records index in descending order
11 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a
12 - Still ascending
13 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a --desc
14 - Now descending

Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 4 Martin Renvoize (ashimema) 2020-10-22 10:44:01 UTC
Works well, Passing QA
Comment 5 Jonathan Druart 2020-11-04 08:53:39 UTC
Created attachment 112976 [details] [review]
Bug 26180: Add descending option to rebuild_elasticsearch.pl

While the ES index is incremental and provides results as it commits, we currently index from the oldest records to the newest.

This patch provides the option to go the other direction

To test:
 1 - Have ES setup and running for Koha
 2 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b
 3 - Note the biblios index from number 1 the end
 4 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a
 5 - Notice the same
 6 - Apply patch
 7 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b
 8 - Still in ascending order
 9 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b --desc
10 - Now records index in descending order
11 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a
12 - Still ascending
13 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a --desc
14 - Now descending

Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

JD amended patch: fix typo "inde" vs "index" and add commit body
Comment 6 Jonathan Druart 2020-11-04 12:03:35 UTC
Pushed to master for 20.11, thanks to everybody involved!
Comment 7 Lucas Gass (lukeg) 2020-11-16 21:16:00 UTC
enhancement will not be backported to 20.05.x