Bug 26051 - Elasticsearch uses the wrong field for callnumber sorting
Summary: Elasticsearch uses the wrong field for callnumber sorting
Status: RESOLVED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching - Elasticsearch (show other bugs)
Version: master
Hardware: All All
: P5 - low normal (vote)
Assignee: Fridolin Somers
QA Contact:
URL:
Keywords:
: 26192 (view as bug list)
Depends on:
Blocks:
 
Reported: 2020-07-23 12:50 UTC by Fridolin Somers
Modified: 2021-03-18 23:48 UTC (History)
11 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Trivial patch
Text to go in the release notes:
This fixes the sorting of search results by call number when using Elasticsearch. Currently it does not sort correctly (uses local-classification instead of cn-sort) and may also cause error messages "No results found" and "Error: Unable to perform your search. Please try again.". This also matches the behaviour used by Zebra.
Version(s) released in:
21.05.00,20.11.04,20.05.10


Attachments
Bug 26051: Elasticsearch field local-classification sort=1 in mapping.yaml (1.67 KB, patch)
2020-07-23 12:55 UTC, Fridolin Somers
Details | Diff | Splinter Review
Bug 26051: Elasticsearch field local-classification sort=1 in mapping.yaml (2.02 KB, patch)
2020-12-27 16:41 UTC, Victor Grousset/tuxayo
Details | Diff | Splinter Review
Bug 26051: [alternate] We should sort callnumber based on cn-sort (2.39 KB, patch)
2021-01-08 13:17 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 26051: Add sort=1 on cn-sort in mappings.yaml (788 bytes, patch)
2021-01-22 08:05 UTC, Fridolin Somers
Details | Diff | Splinter Review
Bug 26051: We should sort callnumber based on cn-sort (2.44 KB, patch)
2021-01-22 08:05 UTC, Fridolin Somers
Details | Diff | Splinter Review
Bug 26051: Add sort=1 on cn-sort in mappings.yaml (836 bytes, patch)
2021-01-30 00:34 UTC, David Nind
Details | Diff | Splinter Review
Bug 26051: We should sort callnumber based on cn-sort (2.49 KB, patch)
2021-01-30 00:34 UTC, David Nind
Details | Diff | Splinter Review
cam12263 - plugin Pando - Ajout d'un plugin pour intégrer les utilisateurs Koha avec Pando (94.10 KB, patch)
2021-02-03 21:01 UTC, Ivan Dziuba
Details | Diff | Splinter Review
Bug 26051: Add sort=1 on cn-sort in mappings.yaml (901 bytes, patch)
2021-02-22 16:15 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 26051: We should sort callnumber based on cn-sort (2.55 KB, patch)
2021-02-22 16:15 UTC, Martin Renvoize
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Fridolin Somers 2020-07-23 12:50:41 UTC
With Elasticsearch the search field use for sorting by callnumber is "local-classification" (from Koha::SearchEngine::Elasticsearch::QueryBuilder::_convert_sort_fields).
So this search field must be sortable in default mappings.yaml.
Comment 1 Fridolin Somers 2020-07-23 12:55:41 UTC
Created attachment 107237 [details] [review]
Bug 26051: Elasticsearch field local-classification sort=1 in mapping.yaml

With Elasticsearch the search field use for sorting by callnumber is "local-classification" (from Koha::SearchEngine::Elasticsearch::QueryBuilder::_convert_sort_fields).
So this search field must be sortable in default mappings.yaml.

If not you see this error in logs :
No mapping found for [local-classification__sort]

Test plan :
1) Install Koha with Elasticsearch with mappings.yaml from sources (admin/searchengine/elasticsearch/mappings.yaml)
2) Reset mappings
3) Rebuild indexes
4) Perform a search
5) Check sorting by callnumber is OK
Comment 2 Victor Grousset/tuxayo 2020-12-27 16:41:06 UTC
Created attachment 114707 [details] [review]
Bug 26051: Elasticsearch field local-classification sort=1 in mapping.yaml

With Elasticsearch the search field use for sorting by callnumber is "local-classification" (from Koha::SearchEngine::Elasticsearch::QueryBuilder::_convert_sort_fields).
So this search field must be sortable in default mappings.yaml.

If not you see this error in logs :
No mapping found for [local-classification__sort]

Test plan :
1) Install Koha with Elasticsearch with mappings.yaml from sources (admin/searchengine/elasticsearch/mappings.yaml)
     It should be the case on all developement instalations
2) Reset mappings (Administration › Search engine configuration (Elasticsearch))
3) Rebuild indexes: misc/search_tools/rebuild_elasticsearch.pl -v -d
4) Perform a search
5) Check sorting by callnumber is OK (add some call numbers if
     necessary)

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Comment 3 Victor Grousset/tuxayo 2020-12-27 16:41:48 UTC
Completed a bit the plan.
Comment 4 Nick Clemens 2021-01-08 13:17:40 UTC
Created attachment 114957 [details] [review]
Bug 26051: [alternate] We should sort callnumber based on cn-sort

In zebra we sort by callnumber using 8007 cn-sort

We should do the same in elasticsearch

To test:
1 - Have Koha using Elasticsearch
2 - Perform a search
3 - Attempt to sort by callnumber
4 - Error in logs: No mapping found for [local-classification__sort]
5 - Apply patch
6 - Restart all
7 - Perform a search and sort by callnumber
8 - Success!
Comment 5 Nick Clemens 2021-01-13 12:44:37 UTC
Victor and Frido, can you look at my alternate patch?
Comment 6 Fridolin Somers 2021-01-21 16:13:17 UTC
I'll do ASAP ;)
Comment 7 Fridolin Somers 2021-01-22 08:05:13 UTC
Created attachment 115662 [details] [review]
Bug 26051: Add sort=1 on cn-sort in mappings.yaml
Comment 8 Fridolin Somers 2021-01-22 08:05:27 UTC
Created attachment 115664 [details] [review]
Bug 26051: We should sort callnumber based on cn-sort

In zebra we sort by callnumber using 8007 cn-sort

We should do the same in elasticsearch

To test:
1 - Have Koha using Elasticsearch
2 - Perform a search
3 - Attempt to sort by callnumber
4 - Error in logs: No mapping found for [local-classification__sort]
5 - Apply patch
6 - Restart all
7 - Perform a search and sort by callnumber
8 - Success!

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Comment 9 Fridolin Somers 2021-01-22 08:07:10 UTC
You patch makes sens :)

I've added a patch to define cn-sort sort=1 in mappings.yaml.
Comment 10 Nick Clemens 2021-01-29 20:22:54 UTC
*** Bug 26192 has been marked as a duplicate of this bug. ***
Comment 11 David Nind 2021-01-30 00:34:39 UTC
Created attachment 116086 [details] [review]
Bug 26051: Add sort=1 on cn-sort in mappings.yaml

Signed-off-by: David Nind <david@davidnind.com>
Comment 12 David Nind 2021-01-30 00:34:43 UTC
Created attachment 116087 [details] [review]
Bug 26051: We should sort callnumber based on cn-sort

In zebra we sort by callnumber using 8007 cn-sort

We should do the same in elasticsearch

To test:
1 - Have Koha using Elasticsearch
2 - Perform a search
3 - Attempt to sort by callnumber
4 - Error in logs: No mapping found for [local-classification__sort]
5 - Apply patch
6 - Restart all
7 - Perform a search and sort by callnumber
8 - Success!

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>

Signed-off-by: David Nind <david@davidnind.com>
Comment 13 David Nind 2021-01-30 00:52:36 UTC
Testing notes (using koha-testing-docker)
- the error message appears in /var/log/koha/kohadev/plack-intranet-error.log 
- enabling Elasticsearch by changing system preference SearchEngine
- package command to reindex: koha-elasticsearch --rebuild -d -b -a -v kohadev
- before applying the patch also got "No results found" and "Error: Unable to perform your search. Please try again." when trying to sort in the staff interface (including the advanced search form) and the OPAC
Comment 14 Ivan Dziuba 2021-02-03 21:01:13 UTC
Created attachment 116295 [details] [review]
cam12263 - plugin Pando - Ajout d'un plugin pour intégrer les utilisateurs Koha avec Pando
Comment 15 David Nind 2021-02-04 06:07:22 UTC
Hi Ivan.

I'm assuming your patch with InLibro's carousel plugin was added in error to this bug?

David
Comment 16 Martin Renvoize 2021-02-22 16:15:38 UTC
Created attachment 117165 [details] [review]
Bug 26051: Add sort=1 on cn-sort in mappings.yaml

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 17 Martin Renvoize 2021-02-22 16:15:41 UTC
Created attachment 117166 [details] [review]
Bug 26051: We should sort callnumber based on cn-sort

In zebra we sort by callnumber using 8007 cn-sort

We should do the same in elasticsearch

To test:
1 - Have Koha using Elasticsearch
2 - Perform a search
3 - Attempt to sort by callnumber
4 - Error in logs: No mapping found for [local-classification__sort]
5 - Apply patch
6 - Restart all
7 - Perform a search and sort by callnumber
8 - Success!

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 18 Martin Renvoize 2021-02-22 16:16:25 UTC
Surprised this one slipped through the net for as long as it has..

Works as expected, no regressions found.

Passing QA

(I also got rid of that final errant attachment)
Comment 19 Jonathan Druart 2021-02-23 12:21:17 UTC
Pushed to master for 21.05, thanks to everybody involved!
Comment 20 Fridolin Somers 2021-02-25 09:36:43 UTC
Pushed to 20.11.x for 20.11.04
Comment 21 Andrew Fuerste-Henry 2021-03-05 13:50:46 UTC
Pushed to 20.05.x for 20.05.10
Comment 22 Victor Grousset/tuxayo 2021-03-18 23:48:23 UTC
Not backported to oldoldstable (19.11.x). Feel free to ask if it's needed.