Summary: | Callnumber sorting is incorrect in Elasticsearch | ||
---|---|---|---|
Product: | Koha | Reporter: | Nick Clemens (kidclamp) <nick> |
Component: | Searching - Elasticsearch | Assignee: | Nick Clemens (kidclamp) <nick> |
Status: | CLOSED FIXED | QA Contact: | Martin Renvoize (ashimema) <martin.renvoize> |
Severity: | normal | ||
Priority: | P5 - low | CC: | arthur.suzuki, bugzilla, david, erica.rohlfs, flyingendpaper, fridolin.somers, lucas, martin.renvoize, tomascohen |
Version: | Main | ||
Hardware: | All | ||
OS: | All | ||
Change sponsored?: | --- | Patch complexity: | Small patch |
Documentation contact: | Documentation submission: | ||
Text to go in the release notes: |
This fixes the sorting of call numbers when using Elasticsearch. Sorting will now work correctly for non-numeric call numbers, for example, E45 will now sort before E7.
|
Version(s) released in: |
22.11.00, 22.05.05
|
Circulation function: | |||
Bug Depends on: | 26051 | ||
Bug Blocks: | |||
Attachments: |
Bug 29632: Don't sort cn-sort numerically
Bug 29632: Don't sort cn-sort numerically Bug 29632: Don't sort cn-sort numerically Bug 29632: Unit tests Bug 29632: Don't sort cn-sort numerically Bug 29632: Unit tests Bug 29632: (QA follow-up) Add ENUM value to kohastructure.sql :-D Bug 29632: (QA follow-up) Fix COMMENT discrepancy on upgrade Bug 29632: [21.11.X] Don't sort cn-sort numerically |
Description
Nick Clemens (kidclamp)
2021-12-03 13:48:05 UTC
Noting that I recreated this workflow on Koha version 20.11. My results returned step 4 as the second result. So, in my testing, Koha returned results in the following order: VA65 E7 T35 1990, VA65 E45 R67 1985, VA65 E7 R63 1984. We're having this problem on 21.05 and can be seen in our OPAC: https://keys.bywatersolutions.com/cgi-bin/koha/opac-search.pl?q=callnum%2Cphr%3Ava65&offset=80&sort_by=call_number_asc You'll see that VA65 E45 is still sorting *after* VA65 E7. It's a big problem for us to continue to have the LC call numbers not sorted correctly--the staff can't sort correctly for paging lists, for example, and being a closed stack library, this impacts us daily, with every service request, where they're operating from Carts, searches, lists, and often generating their own paging/pull lists from various parts of Koha. Created attachment 136682 [details] [review] Bug 29632: Don't sort cn-sort numerically When defining our sort fields in we defined all as 'numeric' For other string containing numbers this is likely correct, however, for callnumbers it is not. e.g. E45 should sort before E7 This patch adds a new 'callnumber' type and deifnes this for cn-sort and adds to the field maping a sort without numeric set To test: 0 - Be using ES with Koha 1 - On records with single item, add callnumbers: VA65 E7 R63 1984 VA65 E7 T35 1990 VA65 E45 R67 1985 2 - Add public note 'shrimp' or something to make them easily searchable as a group 3 - Search for 'shrimp', sort by callnumber 4 - Note E45 comes last, it should come first 5 - Apply patch 6 - Reset ES mappings 7 - Reindex ES 8 - Repeat search 9 - Sorting should be correct when set to callnumber Created attachment 136754 [details] [review] Bug 29632: Don't sort cn-sort numerically When defining our sort fields in we defined all as 'numeric' For other string containing numbers this is likely correct, however, for callnumbers it is not. e.g. E45 should sort before E7 This patch adds a new 'callnumber' type and deifnes this for cn-sort and adds to the field maping a sort without numeric set To test: 0 - Be using ES with Koha 1 - On records with single item, add callnumbers: VA65 E7 R63 1984 VA65 E7 T35 1990 VA65 E45 R67 1985 2 - Add public note 'shrimp' or something to make them easily searchable as a group 3 - Search for 'shrimp', sort by callnumber 4 - Note E45 comes last, it should come first 5 - Apply patch 6 - Reset ES mappings 7 - Reindex ES 8 - Repeat search 9 - Sorting should be correct when set to callnumber Signed-off-by: David Nind <david@davidnind.com> Testing notes (using koha-testing-docker): - After applying the patch (step 5) you need to run a database update (updatedatabase0 Created attachment 137182 [details] [review] Bug 29632: Don't sort cn-sort numerically When defining our sort fields in we defined all as 'numeric' For other string containing numbers this is likely correct, however, for callnumbers it is not. e.g. E45 should sort before E7 This patch adds a new 'callnumber' type and deifnes this for cn-sort and adds to the field maping a sort without numeric set To test: 0 - Be using ES with Koha 1 - On records with single item, add callnumbers: VA65 E7 R63 1984 VA65 E7 T35 1990 VA65 E45 R67 1985 2 - Add public note 'shrimp' or something to make them easily searchable as a group 3 - Search for 'shrimp', sort by callnumber 4 - Note E45 comes last, it should come first 5 - Apply patch 6 - Reset ES mappings 7 - Reindex ES 8 - Repeat search 9 - Sorting should be correct when set to callnumber Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Michal Urban <michalurban177@gmail.com> Looks like we're missing a unit test here for the Koha/SearchEngine/Elasticsearch.pm change. Created attachment 137539 [details] [review] Bug 29632: Unit tests This patch adds unit tests, as well as changing existing test to use a mock and read the data as passed in tests, rather than relying on what exists in the DB Created attachment 137605 [details] [review] Bug 29632: Don't sort cn-sort numerically When defining our sort fields in we defined all as 'numeric' For other string containing numbers this is likely correct, however, for callnumbers it is not. e.g. E45 should sort before E7 This patch adds a new 'callnumber' type and deifnes this for cn-sort and adds to the field maping a sort without numeric set To test: 0 - Be using ES with Koha 1 - On records with single item, add callnumbers: VA65 E7 R63 1984 VA65 E7 T35 1990 VA65 E45 R67 1985 2 - Add public note 'shrimp' or something to make them easily searchable as a group 3 - Search for 'shrimp', sort by callnumber 4 - Note E45 comes last, it should come first 5 - Apply patch 6 - Reset ES mappings 7 - Reindex ES 8 - Repeat search 9 - Sorting should be correct when set to callnumber Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Michal Urban <michalurban177@gmail.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Created attachment 137606 [details] [review] Bug 29632: Unit tests This patch adds unit tests, as well as changing existing test to use a mock and read the data as passed in tests, rather than relying on what exists in the DB Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Thanks Nick, Passing QA now :) Shouldn't we add the index to the configuration on the update? Pushed to master for 22.11. Nice work everyone, thanks! Created attachment 137823 [details] [review] Bug 29632: (QA follow-up) Add ENUM value to kohastructure.sql :-D Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Created attachment 138611 [details] [review] Bug 29632: (QA follow-up) Fix COMMENT discrepancy on upgrade Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Backported to 22.05.x for 22.05.05 conflicts in templates when trying to apply on 21.11. Provide a backport if needed. Created attachment 139827 [details] [review] Bug 29632: [21.11.X] Don't sort cn-sort numerically When defining our sort fields in we defined all as 'numeric' For other string containing numbers this is likely correct, however, for callnumbers it is not. e.g. E45 should sort before E7 This patch adds a new 'callnumber' type and deifnes this for cn-sort and adds to the field maping a sort without numeric set To test: 0 - Be using ES with Koha 1 - On records with single item, add callnumbers: VA65 E7 R63 1984 VA65 E7 T35 1990 VA65 E45 R67 1985 2 - Add public note 'shrimp' or something to make them easily searchable as a group 3 - Search for 'shrimp', sort by callnumber 4 - Note E45 comes last, it should come first 5 - Apply patch 6 - Reset ES mappings 7 - Reindex ES 8 - Repeat search 9 - Sorting should be correct when set to callnumber Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Michal Urban <michalurban177@gmail.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Bug 29632: Unit tests This patch adds unit tests, as well as changing existing test to use a mock and read the data as passed in tests, rather than relying on what exists in the DB Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Bug 29632: (QA follow-up) Add ENUM value to kohastructure.sql :-D Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Bug 29632: (QA follow-up) Fix COMMENT discrepancy on upgrade Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> (In reply to Arthur Suzuki from comment #17) > conflicts in templates when trying to apply on 21.11. > Provide a backport if needed. Patch supplied |