Summary: | Authorities 'starts with' search returns the same as 'contains' when using ICU | ||
---|---|---|---|
Product: | Koha | Reporter: | Colin Campbell <colin.campbell> |
Component: | Searching | Assignee: | David Cook <dcook> |
Status: | CLOSED FIXED | QA Contact: | Martin Renvoize (ashimema) <martin.renvoize> |
Severity: | normal | ||
Priority: | P5 - low | CC: | david, dcook, fridolin.somers, gaetan.boisson, janet.mcgowan, marjorie.barry-vila, martin.renvoize, ztajoli |
Version: | Main | ||
Hardware: | All | ||
OS: | All | ||
See Also: |
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13751 https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27299 |
||
Change sponsored?: | --- | Patch complexity: | Trivial patch |
Documentation contact: | Documentation submission: | ||
Text to go in the release notes: |
This fixes searching authorities when using ICU* so that 'starts with' searching works correctly. Before this, a 'starts with' search would return the same results as 'contains'.
Technical details: this adds the "complete field" to the authority "starts with" search so that it uses the untokenized "p" register.
(* ICU is a feature of the Zebra search engine that can be configured to make searching with non-latin languages (such as Chinese and Arabic) work correctly.)
|
Version(s) released in: |
22.11.00
|
Circulation function: | |||
Bug Depends on: | 27299 | ||
Bug Blocks: | |||
Attachments: |
Proposed patch
Bug 11158: Add complete field to authority "starts with" search Bug 11158: Add complete field to authority "starts with" search Bug 11158: Add complete field to authority "starts with" search |
Description
Colin Campbell
2013-10-29 12:40:16 UTC
Created attachment 22517 [details] [review] Proposed patch This change appears to fix the problem. At the same time it does not appear to break the behaviour with 'chr' To test on a system set up to use ICU find a test case similar to that in the bug description. Apply the patch and redo the 'starts with' search. There is also a question over the rule being used to tokenise phrase searches and what would be the best default configuration (including no rule) further investigation is dependent on bug 10729 as phrase and keyword searching have different requirements in this respect Hi, I have test it on a system with ICU and with the patch of bug 10729. The result is this error on Opac: ZOOM error 20003 "can't set prefix query" (addinfo: "@or @attr 7=1 @attr 1=Heading 0 @and @attr 1=authtype @attr 5=100 TOPIC_TERM @attr 1=Heading \@attr 3=1 \@attr 6=3 \@attr 4=1 \@attr 5=1 "Biblia"") from diag-set 'ZOOM'</pre> Same error in Intranet auth search. Do you have a proposal for use together this patch and bug 10729 patch ? I've managed to reproduce Zeno's result and I'm investigating. The problem seems to be occuring irrespective to whether or not bug 10729 is applied, (which is logical looking at the error return) The search part of the manual states that a 'begins with' search is not possible with ICU - this here seems a bit related. Is there way we can make this work? (In reply to Colin Campbell from comment #2) > There is also a question over the rule being used to tokenise phrase > searches and what would be the best default configuration (including no > rule) further investigation is dependent on bug 10729 as phrase and keyword > searching have different requirements in this respect I cover this more at https://github.com/indexdata/idzebra/issues/24#issuecomment-751260285 but yeah the tokenization is definitely the issue I hadn't thought of the complete field attribute and truncation... interesting. For what it's worth, without the @attr 6=3, it uses the "w" register: https://software.indexdata.com/zebra/doc/querymodel-zebra.html#querymodel-pqf-apt-mapping-structuretype Due to my work on Bug 27299, I have renewed interest in this one. In koha-testing-docker, the following URL http://localhost:8081/cgi-bin/koha/authorities/authorities-home.pl?op=do_search&type=intranet&authtypecode=&marclist=mainmainentry&and_or=and&excluding=&operator=start&value=Espen&orderby=HeadingAsc will generate the following Zebra query: @attrset Bib-1 @or @attr 1=Heading @attr 7=1 0 @attr 5=1 @attr 4=1 @attr 3=2 @attr 1=Heading-Main Espen The key part of that is the following: @attr 5=1 @attr 4=1 @attr 3=2 @attr 1=Heading-Main Espen Breaking that down: @attr 5=1 -> Right truncation @attr 4=1 -> Phrase structure @attr 3=2 -> Position first in subfield @attr 1=Heading-Main Espen As Colin notes, the key is really the addition of a @attr 6=3 to force a complete field search, which uses the "p" register. In my testing, we can leave @attr 3=2 as is, since it doesn't really make a difference. But adding @attr 6=3 fixes the issue - at least when the patch from Bug 27299 is applied... Colin's patch doesn't apply anymore, so I'm going to do a new one, which just adds @attr 6=3. I'm also adding a dependency on Bug 27299 since that'll be crucial Created attachment 114829 [details] [review] Bug 11158: Add complete field to authority "starts with" search This patch adds "complete field" to the authority "starts with" search so that it uses the untokenized "p" register. Test plan: 1. Apply the patch 2. koha-plack --restart kohadev 3. Go to http://localhost:8081/cgi-bin/koha/authorities/authorities-home.pl 4. Type in "Espen" into the search box and hit "Submit" 5. Note that there are 3 results 6. Change "contains" to "starts with" and hit "Submit" 7. Note that no results are returned 8. Change the search from "Espen" to "Sandberg" and hit "Submit" 9. Note that 3 results are returned 10. Experiment to your heart's content and rejoice at your new found power The test plan works ok on current master for me, but I assume this is because the kohadevbox is using ICU by default. I am not sure how to activate ICU there. (In reply to Katrin Fischer from comment #12) > The test plan works ok on current master for me, but I assume this is > because the kohadevbox is using ICU by default. I am not sure how to > activate ICU there. I think you mean using CHR by default. I wonder if we should change to ICU default some time. It involves updating /etc/koha/zebradb/etc/default.idx and then re-indexing Zebra. Although I didn't have that in my test plan I wonder why... Noticed this as one of the "10 Oldest bugs" under "Needs Signoff". Figured I'd update to the Assignee to me too since I apparently wrote the patch 1.5 years ago... Created attachment 137049 [details] [review] Bug 11158: Add complete field to authority "starts with" search This patch adds "complete field" to the authority "starts with" search so that it uses the untokenized "p" register. Test plan: 1. Apply the patch 2. koha-plack --restart kohadev 3. Go to http://localhost:8081/cgi-bin/koha/authorities/authorities-home.pl 4. Type in "Espen" into the search box and hit "Submit" 5. Note that there are 3 results 6. Change "contains" to "starts with" and hit "Submit" 7. Note that no results are returned 8. Change the search from "Espen" to "Sandberg" and hit "Submit" 9. Note that 3 results are returned 10. Experiment to your heart's content and rejoice at your new found power Signed-off-by: David Nind <david@davidnind.com> Testing notes (using koha-testing-docker): 1. This is my first time setting up ICU - I followed the instructions at https://wiki.koha-community.org/wiki/ICU_chains_configuration except for enabling system preference UseICU, as it no longer exists 2. Queries about setting up ICU: . Are the instructions still current? . There is no system preference UseICU, but there is UseICUStyleQuotes . When editing /etc/koha/zebradb/etc/default.idx I noticed the URX (URL) index, Numeric index, and Sort register sections reference *.chr files - do these need changing as well? 3. I used the query in the description (Smith) as well when testing. Created attachment 137699 [details] [review] Bug 11158: Add complete field to authority "starts with" search This patch adds "complete field" to the authority "starts with" search so that it uses the untokenized "p" register. Test plan: 1. Apply the patch 2. koha-plack --restart kohadev 3. Go to http://localhost:8081/cgi-bin/koha/authorities/authorities-home.pl 4. Type in "Espen" into the search box and hit "Submit" 5. Note that there are 3 results 6. Change "contains" to "starts with" and hit "Submit" 7. Note that no results are returned 8. Change the search from "Espen" to "Sandberg" and hit "Submit" 9. Note that 3 results are returned 10. Experiment to your heart's content and rejoice at your new found power Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Nice one David and David!.. It's great to see Zebra having a little love still.. All works as described, so great.. Passing QA (In reply to Martin Renvoize from comment #18) > Nice one David and David!.. > > It's great to see Zebra having a little love still.. I'll keep showing it the love until all my libraries jump ship to Elasticsearch! Pushed to master for 22.11. Nice work everyone, thanks! |