Summary: | ICU does not strip = when indexing/searching | ||
---|---|---|---|
Product: | Koha | Reporter: | David Cook <dcook> |
Component: | Searching - Zebra | Assignee: | David Cook <dcook> |
Status: | RESOLVED FIXED | QA Contact: | Martin Renvoize (ashimema) <martin.renvoize> |
Severity: | normal | ||
Priority: | P5 - low | CC: | fridolin.somers, lucas, wainuiwitikapark |
Version: | Main | ||
Hardware: | All | ||
OS: | All | ||
GIT URL: | Change sponsored?: | --- | |
Patch complexity: | --- | Documentation contact: | |
Documentation submission: | Text to go in the release notes: |
This change fixes an issue with Zebra ICU searching where titles with colons aren't properly searchable, especially when used with Analytics.
A full re-index of Zebra is needed for this change to take effect.
|
|
Version(s) released in: |
24.05.00,23.11.02,23.05.08
|
Circulation function: | |
Attachments: |
Bug 35455: Remove = in ICU for indexing/searching
Record 1 Record 2 for testing Bug 35455: Remove = in ICU for indexing/searching Bug 35455: Remove = in ICU for indexing/searching |
Description
David Cook
2023-12-01 01:50:40 UTC
For example: Record 1: 245$a Awesome title: awesome subtitle Record 2: 245$a Cool article 773$t Awesome title: awesome subtitle -- At indexing time, this will be indexed as "Awesome title awesome subtitle" as the ICU removes the punctuation. At search time, we can't use "Awesome title= awesome subtitle" because the tokens "title" and "title=" won't match. We've got to remove the = since at indexing time it was a : Created attachment 159422 [details] [review] Bug 35455: Remove = in ICU for indexing/searching This change more closely aligns ICU and CHR so that ICU also removes the = character. This fixes issues in ICU when searching with a : which gets transformed into a =. Without this change, the Analytics features won't work for titles with a colon in them. Test plan: 0. Apply the patch and import bibs from Bugzilla (using Staged MARC tools) 1. cp ./etc/zebradb/etc/phrases-icu.xml /etc/koha/zebradb/etc/phrases-icu.xml 2. cp ./etc/zebradb/etc/words-icu.xml /etc/koha/zebradb/etc/words-icu.xml 3. vi /etc/koha/zebradb/etc/default.idx Change "charmap word-phrase-utf.chr" to "icuchain words-icu.xml" for "index w" and "icuchain phrases-icu.xml" for "index p" 4. koha-zebra --stop kohadev 5. pkill zebrasrv 6. koha-zebra --start kohadev 7. koha-rebuild-zebra -a -b -f -v kohadev 8. Search for "Awesome title" and open the detail page 9. Note that the "Analytics: Show analytics" line shows up 10. Click that link 11. Note that it opens the "Cool article" record and it displays "In: Awesome title: awesome subtitle" 12. Click that link 13. Note that it opens the "Awesome title" record Created attachment 159423 [details]
Record 1
Record 1 for testing
Created attachment 159424 [details]
Record 2 for testing
Record 2 for testing
Created attachment 159512 [details] [review] Bug 35455: Remove = in ICU for indexing/searching This change more closely aligns ICU and CHR so that ICU also removes the = character. This fixes issues in ICU when searching with a : which gets transformed into a =. Without this change, the Analytics features won't work for titles with a colon in them. Test plan: 0. Apply the patch and import bibs from Bugzilla (using Staged MARC tools) 1. cp ./etc/zebradb/etc/phrases-icu.xml /etc/koha/zebradb/etc/phrases-icu.xml 2. cp ./etc/zebradb/etc/words-icu.xml /etc/koha/zebradb/etc/words-icu.xml 3. vi /etc/koha/zebradb/etc/default.idx Change "charmap word-phrase-utf.chr" to "icuchain words-icu.xml" for "index w" and "icuchain phrases-icu.xml" for "index p" 4. koha-zebra --stop kohadev 5. pkill zebrasrv 6. koha-zebra --start kohadev 7. koha-rebuild-zebra -a -b -f -v kohadev 8. Search for "Awesome title" and open the detail page 9. Note that the "Analytics: Show analytics" line shows up 10. Click that link 11. Note that it opens the "Cool article" record and it displays "In: Awesome title: awesome subtitle" 12. Click that link 13. Note that it opens the "Awesome title" record Signed-off-by: David Nind <david@davidnind.com> Created attachment 160011 [details] [review] Bug 35455: Remove = in ICU for indexing/searching This change more closely aligns ICU and CHR so that ICU also removes the = character. This fixes issues in ICU when searching with a : which gets transformed into a =. Without this change, the Analytics features won't work for titles with a colon in them. Test plan: 0. Apply the patch and import bibs from Bugzilla (using Staged MARC tools) 1. cp ./etc/zebradb/etc/phrases-icu.xml /etc/koha/zebradb/etc/phrases-icu.xml 2. cp ./etc/zebradb/etc/words-icu.xml /etc/koha/zebradb/etc/words-icu.xml 3. vi /etc/koha/zebradb/etc/default.idx Change "charmap word-phrase-utf.chr" to "icuchain words-icu.xml" for "index w" and "icuchain phrases-icu.xml" for "index p" 4. koha-zebra --stop kohadev 5. pkill zebrasrv 6. koha-zebra --start kohadev 7. koha-rebuild-zebra -a -b -f -v kohadev 8. Search for "Awesome title" and open the detail page 9. Note that the "Analytics: Show analytics" line shows up 10. Click that link 11. Note that it opens the "Cool article" record and it displays "In: Awesome title: awesome subtitle" 12. Click that link 13. Note that it opens the "Awesome title" record Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> All works as described and makes sense. Passing QA Pushed for 24.05! Well done everyone, thank you! Pushed to 23.11.x for 23.11.02 Backported to 23.05.x for upcoming 23.05.08 Not backporting to 22.11 |