Summary: | Sorting results by Title A-Z might use wrong title field | ||
---|---|---|---|
Product: | Koha | Reporter: | Katrin Fischer <katrin.fischer> |
Component: | Searching - Elasticsearch | Assignee: | Nick Clemens (kidclamp) <nick> |
Status: | Needs documenting --- | QA Contact: | Katrin Fischer <katrin.fischer> |
Severity: | normal | ||
Priority: | P5 - low | CC: | david, dcook, januszop, matt.blenkinsop, nick |
Version: | Main | ||
Hardware: | All | ||
OS: | All | ||
See Also: | https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26472 | ||
GIT URL: | Change sponsored?: | --- | |
Patch complexity: | Small patch | Documentation contact: | |
Documentation submission: | Text to go in the release notes: |
This changes the behavour of title search and sorting (Title (A-Z)) in the OPAC and staff interface when using Elasticsearch or Open Search. The title search now only uses 245 (for MARC 21) and 200 (for UNIMARC). Previously other title fields may have affected the search order, for example 240$a in MARC21.
To make this change for existing installations, reset the search engine mappings (Administration > Catalog > Search engine configuration (Elasticsearch) > Reset mappings (scroll down to the bottom of the page)) and rebuild the search index (koha-elasticsearch --rebuild -d -b -a <instancename>). If you have customized the search engine configuration, remember to record or back these up BEFORE resetting the mappings.
|
|
Version(s) released in: |
23.05.00
|
Circulation function: | |
Bug Depends on: | 33206 | ||
Bug Blocks: | |||
Attachments: |
Bug 33594: Update mappings and comment
Bug 33594: Only sort on title main heading Bug 33594: Update mappings and comment Bug 33594: Only sort on title main heading Bug 33594: Only sort on title main heading |
Description
Katrin Fischer
2023-04-23 21:40:03 UTC
I have encountered this some time ago. For me worked customizing the mappings.yaml as follows: - facet: '' marc_field: 240 marc_type: marc21 - sort: ~ + sort: '' suggestible: '' I.e. changing ~ into '' for all MARC fields except 245. For some to-me-mysterious reason, ~ (= undef) is explicitly interpreted as "yes, include it into the sort search field" by Koha/SearchEngine/Elasticsearch.pm: # Sort is a bit special as it can be true, false, undef. # We care about "true" or "undef", # "undef" means to do the default thing, which is make it sortable. So, since this is by design and mappings.yaml is configurable, I didn't treat it as a bug. Just use a customized copy of mappings. Please, check if changing ~ into '' works for you. Please, consider also if it is worth to be a bug. If so, the solution is there. (In reply to Janusz Kaczmarek from comment #1) > > I.e. changing ~ into '' for all MARC fields except 245. > I meant, for title search field, of course. (In reply to Janusz Kaczmarek from comment #2) > (In reply to Janusz Kaczmarek from comment #1) > > > > I.e. changing ~ into '' for all MARC fields except 245. > > > > I meant, for title search field, of course. Hi Janusz, thx for the insight, this is really helpful. I also found bug 32479. Would you agree that all ~ shoudl be replaced by "" or 0? I feel it's a bug and we should fix it, but still a little confused about the different values we could use here for sort and what effect they will have. Can someone explain a little more? (In reply to Katrin Fischer from comment #4) > I feel it's a bug and we should fix it, but still a little confused about > the different values we could use here for sort and what effect they will > have. Can someone explain a little more? 27316 moved the options for setting the field in staff interface from "Undef/Yes/No" to simply "Yes/No" = 0/1 We did not update the mappings file, that should be done here The code continues to support ~/Undef so that we don't break existing setups - we should update the comment Janusz noted to explain that 'UNdef' should be set as '1' Created attachment 150649 [details] [review] Bug 33594: Update mappings and comment This changes a few other fields, but it really is just a dump from the DB, those changes are essentially how the fields are going to be treated anyways, it was generated using: perl misc/search_tools/export_elasticsearch_mappings.pl > admin/searchengine/elasticsearch/mappings.yaml Nick, thank you for your contribution. Changing "sort: ~" into "sort: 1" makes the mappings.yaml to be more explicit and therefor less mysterious. But the patch does not change the way Koha with ES sorts titles, so it does not fix the problem. I think, everybody would expect that when choosing sort order by title (both in staff and OPAC interface) the sorting will by done according to the main title, i.e. 245 in MARC 21 and 200 in UNIMARC. So, IMHO, in searchfield 'title' only 245 (for MARC 21) and 200 (for UNIMARC) should have 'sort: 1', and the rest of tags indexed with title -- 'sort: 0'. I would appreciate your feedback. (In reply to Janusz Kaczmarek from comment #8) > Nick, thank you for your contribution. Changing "sort: ~" into "sort: 1" > makes the mappings.yaml to be more explicit and therefor less mysterious. > > But the patch does not change the way Koha with ES sorts titles, so it does > not fix the problem. I think, everybody would expect that when choosing > sort order by title (both in staff and OPAC interface) the sorting will by > done according to the main title, i.e. 245 in MARC 21 and 200 in UNIMARC. > > So, IMHO, in searchfield 'title' only 245 (for MARC 21) and 200 (for > UNIMARC) should have 'sort: 1', and the rest of tags indexed with title -- > 'sort: 0'. > > I would appreciate your feedback. I'd agree with that expectation. The other titles don't even all show, so it can get quite confusing. (In reply to Janusz Kaczmarek from comment #8) > Nick, thank you for your contribution. Changing "sort: ~" into "sort: 1" > makes the mappings.yaml to be more explicit and therefor less mysterious. > > But the patch does not change the way Koha with ES sorts titles, so it does > not fix the problem. I think, everybody would expect that when choosing > sort order by title (both in staff and OPAC interface) the sorting will by > done according to the main title, i.e. 245 in MARC 21 and 200 in UNIMARC. > > So, IMHO, in searchfield 'title' only 245 (for MARC 21) and 200 (for > UNIMARC) should have 'sort: 1', and the rest of tags indexed with title -- > 'sort: 0'. > > I would appreciate your feedback. Agreed - it looks like the Elastic default on a multi-value sort field is min (ascending) or max (descending) Created attachment 150749 [details] [review] Bug 33594: Only sort on title main heading This patch simply remvoes sort from all elements that are not strictly the main title Note: If multiple fields are set as sort, they are collapsed into a single entry in the {field}__sort field in the ES index. The order will be determined by the order in the marc record To test: 1 - Apply patch 2 - perl misc/search_tools/rebuild_elasticsearch -r -v 3 - Search the catalog 4 - Sort by title 5 - Confirm records are correct 6 - Add a 240 (before the 245) with subfield a 'AAAAA' 7 - Confirm sorting is not affected 8 - View record details, click 'Elasticsearch record: Show' 9 - Find 'title__sort' and confirm it looks correct (does not include AAAAA) Created attachment 150792 [details] [review] Bug 33594: Update mappings and comment Signed-off-by: David Nind <david@davidnind.com> Created attachment 150793 [details] [review] Bug 33594: Only sort on title main heading This patch simply remvoes sort from all elements that are not strictly the main title Note: If multiple fields are set as sort, they are collapsed into a single entry in the {field}__sort field in the ES index. The order will be determined by the order in the marc record To test: 1 - Apply patch 2 - perl misc/search_tools/rebuild_elasticsearch -r -v 3 - Search the catalog 4 - Sort by title 5 - Confirm records are correct 6 - Add a 240 (before the 245) with subfield a 'AAAAA' 7 - Confirm sorting is not affected 8 - View record details, click 'Elasticsearch record: Show' 9 - Find 'title__sort' and confirm it looks correct (does not include AAAAA) Signed-off-by: David Nind <david@davidnind.com> Testing notes (using KTD): 1. I used the example in the bug description (#c0) to test. 2. After step 1, I needed to reset the mappings (Administration > Catalog > Search engine configuration (Elasticsearch) > Reset mappings (scroll down to the bottom of the page)). 3. Draft release notes. I've made an attempt at this, but this area is not something I really understand. Please update to fix what I've got wrong! Created attachment 150813 [details] [review] Bug 33594: Only sort on title main heading This patch simply remvoes sort from all elements that are not strictly the main title Note: If multiple fields are set as sort, they are collapsed into a single entry in the {field}__sort field in the ES index. The order will be determined by the order in the marc record To test: 1 - Apply patch 2 - perl misc/search_tools/rebuild_elasticsearch -r -v 3 - Search the catalog 4 - Sort by title 5 - Confirm records are correct 6 - Add a 240 (before the 245) with subfield a 'AAAAA' 7 - Confirm sorting is not affected 8 - View record details, click 'Elasticsearch record: Show' 9 - Find 'title__sort' and confirm it looks correct (does not include AAAAA) Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Pushed to master for 23.05. Nice work everyone, thanks! Enhancement - not backporting to 22.11.x It would be nice to have this reconsidered, but it will only affect new installations. |