| Summary: | Find duplicate removes operators from the middle of search terms | ||
|---|---|---|---|
| Product: | Koha | Reporter: | Nick Clemens (kidclamp) <nick> |
| Component: | Searching | Assignee: | Nick Clemens (kidclamp) <nick> |
| Status: | CLOSED FIXED | QA Contact: | Testopia <testopia> |
| Severity: | normal | ||
| Priority: | P5 - low | CC: | andrew, caroline.cyr-la-rose, fridolin.somers, margaret, martin.renvoize, severine.queune, victor |
| Version: | Main | ||
| Hardware: | All | ||
| OS: | All | ||
| Change sponsored?: | --- | Patch complexity: | Trivial patch |
| Documentation contact: | Documentation submission: | ||
| Text to go in the release notes: | Version(s) released in: |
21.05.00,20.11.02,20.05.09
|
|
| Circulation function: | |||
| Attachments: |
Bug 26957: Unit tests
Bug 26957: Don't remove words that are also operators from search terms Bug 26957: Don't remove words that are also operators from search terms Test records Bug 26957: Don't remove words that are also operators from search terms Bug 26957: Unit tests Bug 26957: Don't remove words that are also operators from search terms |
||
Created attachment 113933 [details] [review] Bug 26957: Unit tests Created attachment 113934 [details] [review] Bug 26957: Don't remove words that are also operators from search terms FindDuplicate has a hardcoded regex to remove 'and', 'or', and 'not' from any search fields, however, is does this with no regard for these terms being embedded in the middle of a word or phrase. E.g.: 'Coupland' becomes 'Coupl' 'Frog and toad' becomes 'frog toad' etc. This patch simplay removes the substitutions as we already quote the terms as well To test: 1 - Add records to your system like: The night circus by Erin Morgenstern Frog and toad all year Nothing ever happens 2 - Attempt to add purchase suggestions like title: the night circus, author:Morgenstern, Erin title: For and toad all year title:nothing ever happens (case is important) 3 - No warnings about existing biblios 4 - Apply patches 5 - Delete existing suggestions 6 - Repeat step 2 7 - Each attempt should warn you of existing biblio Created attachment 113935 [details] [review] Bug 26957: Don't remove words that are also operators from search terms FindDuplicate has a hardcoded regex to remove 'and', 'or', and 'not' from any search fields, however, is does this with no regard for these terms being embedded in the middle of a word or phrase. E.g.: 'Coupland' becomes 'Coupl' 'Frog and toad' becomes 'frog toad' etc. This patch simplay removes the substitutions as we already quote the terms as well To test: 1 - Add records to your system like: The night circus by Erin Morgenstern Frog and toad all year Nothing ever happens 2 - Attempt to add purchase suggestions like title: the night circus, author:Morgenstern, Erin title: For and toad all year title:nothing ever happens (case is important) 3 - No warnings about existing biblios 4 - Apply patches 5 - Delete existing suggestions 6 - Repeat step 2 7 - Each attempt should warn you of existing biblio Created attachment 113936 [details]
Test records
Created attachment 114716 [details] [review] Bug 26957: Don't remove words that are also operators from search terms FindDuplicate has a hardcoded regex to remove 'and', 'or', and 'not' from any search fields, however, is does this with no regard for these terms being embedded in the middle of a word or phrase. E.g.: 'Coupland' becomes 'Coupl' 'Frog and toad' becomes 'frog toad' etc. This patch simplay removes the substitutions as we already quote the terms as well To test: 1 - Add records to your system like: The night circus by Erin Morgenstern Frog and toad all year Nothing ever happens 2 - Attempt to add purchase suggestions like title: the night circus, author:Morgenstern, Erin title: For and toad all year title:nothing ever happens (case is important) 3 - No warnings about existing biblios 4 - Apply patches 5 - Delete existing suggestions 6 - Repeat step 2 7 - Each attempt should warn you of existing biblio Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com> Whaou what an ugly bug ^^. I confirm finding duplicate works even with title containing operators like 'and'. For example 'War and'. Created attachment 115079 [details] [review] Bug 26957: Unit tests Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Created attachment 115080 [details] [review] Bug 26957: Don't remove words that are also operators from search terms FindDuplicate has a hardcoded regex to remove 'and', 'or', and 'not' from any search fields, however, is does this with no regard for these terms being embedded in the middle of a word or phrase. E.g.: 'Coupland' becomes 'Coupl' 'Frog and toad' becomes 'frog toad' etc. This patch simplay removes the substitutions as we already quote the terms as well To test: 1 - Add records to your system like: The night circus by Erin Morgenstern Frog and toad all year Nothing ever happens 2 - Attempt to add purchase suggestions like title: the night circus, author:Morgenstern, Erin title: For and toad all year title:nothing ever happens (case is important) 3 - No warnings about existing biblios 4 - Apply patches 5 - Delete existing suggestions 6 - Repeat step 2 7 - Each attempt should warn you of existing biblio Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> What a crazy bug! Fix works well and no regressions found. Passing QA Pushed to master for 21.05, thanks to everybody involved! Pushed to 20.11.x for 20.11.02 Pushed to 20.05.x for 20.05.09 Not backported to oldoldstable (19.11.x). Feel free to ask if it's needed. |
To test: 1 - Edit your koha-conf.xml and set zebra debug levels to include request <zebra_loglevels>none,fatal,warn,request</zebra_loglevels> 2 - Restart all the things 3 - Tail the zebra output log tail -f /var/log/koha/kohadev/zebra-output.log 4 - Try to create a new purchase suggestion for the author: Morgenstern 5 - Note the logged search is for Mgenstern