Bug 26180

Summary: Elasticsearch - Add option to index records in descending order
Product: Koha Reporter: Nick Clemens (kidclamp) <nick>
Component: Searching - ElasticsearchAssignee: Nick Clemens (kidclamp) <nick>
Status: CLOSED FIXED QA Contact:
Severity: enhancement    
Priority: P5 - low CC: jonathan.druart, lucas, martin.renvoize
Version: Main   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
20.11.00
Attachments: Bug 26180: Add descending option to rebuild_elasticsearch.pl
Bug 26180: Add descending option to rebuild_elasticsearch.pl
Bug 26180: Add descending option to rebuild_elasticsearch.pl
Bug 26180: Add descending option to rebuild_elasticsearch.pl

Description Nick Clemens (kidclamp) 2020-08-07 13:14:27 UTC
While the ES index is incremental and provides results as it commits, we currently index from the oldest records to the newest.

It would be nice to provide the option to go the other direction
Comment 1 Nick Clemens (kidclamp) 2020-08-07 13:21:07 UTC
Created attachment 107955 [details] [review]
Bug 26180: Add descending option to rebuild_elasticsearch.pl

TO test:
 1 - Have ES setup and running for Koha
 2 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b
 3 - Note the biblios index from number 1 the end
 4 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a
 5 - Notice the same
 6 - Apply patch
 7 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b
 8 - Still in ascending order
 9 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b --desc
10 - Now records index in descending order
11 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a
12 - Still ascending
13 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a --desc
14 - Now descending
Comment 2 Séverine Queune 2020-10-22 08:07:25 UTC
Created attachment 112146 [details] [review]
Bug 26180: Add descending option to rebuild_elasticsearch.pl

TO test:
 1 - Have ES setup and running for Koha
 2 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b
 3 - Note the biblios index from number 1 the end
 4 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a
 5 - Notice the same
 6 - Apply patch
 7 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b
 8 - Still in ascending order
 9 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b --desc
10 - Now records index in descending order
11 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a
12 - Still ascending
13 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a --desc
14 - Now descending

Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>
Comment 3 Martin Renvoize 2020-10-22 10:43:41 UTC
Created attachment 112159 [details] [review]
Bug 26180: Add descending option to rebuild_elasticsearch.pl

TO test:
 1 - Have ES setup and running for Koha
 2 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b
 3 - Note the biblios index from number 1 the end
 4 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a
 5 - Notice the same
 6 - Apply patch
 7 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b
 8 - Still in ascending order
 9 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b --desc
10 - Now records index in descending order
11 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a
12 - Still ascending
13 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a --desc
14 - Now descending

Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 4 Martin Renvoize 2020-10-22 10:44:01 UTC
Works well, Passing QA
Comment 5 Jonathan Druart 2020-11-04 08:53:39 UTC
Created attachment 112976 [details] [review]
Bug 26180: Add descending option to rebuild_elasticsearch.pl

While the ES index is incremental and provides results as it commits, we currently index from the oldest records to the newest.

This patch provides the option to go the other direction

To test:
 1 - Have ES setup and running for Koha
 2 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b
 3 - Note the biblios index from number 1 the end
 4 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a
 5 - Notice the same
 6 - Apply patch
 7 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b
 8 - Still in ascending order
 9 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -b --desc
10 - Now records index in descending order
11 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a
12 - Still ascending
13 - perl misc/search_tools/rebuild_elasticsearch.pl -v -v -a --desc
14 - Now descending

Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

JD amended patch: fix typo "inde" vs "index" and add commit body
Comment 6 Jonathan Druart 2020-11-04 12:03:35 UTC
Pushed to master for 20.11, thanks to everybody involved!
Comment 7 Lucas Gass 2020-11-16 21:16:00 UTC
enhancement will not be backported to 20.05.x