Bug 22787 - Mapping missing for ů to u in word-phrase-utf-chr
Summary: Mapping missing for ů to u in word-phrase-utf-chr
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal
Assignee: Josef Moravec
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-04-26 17:48 UTC by Nick Clemens (kidclamp)
Modified: 2020-06-04 20:32 UTC (History)
6 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
19.05.00, 18.11.06
Circulation function:


Attachments
Bug 22787: Add mapping of letters "ů" and "Ů" to zebra definitions (1.37 KB, patch)
2019-04-29 12:34 UTC, Josef Moravec
Details | Diff | Splinter Review
Bug 22787: Add mapping of letters "ů" and "Ů" to zebra definitions (1.41 KB, patch)
2019-05-02 18:49 UTC, Michal Denar
Details | Diff | Splinter Review
Bug 22787: Add mapping of letters "ů" and "Ů" to zebra definitions (1.47 KB, patch)
2019-05-04 07:51 UTC, Chris Cormack
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Comment 1 Josef Moravec 2019-04-29 12:16:14 UTC
(In reply to Nick Clemens from comment #0)
> The problem was noticed when searching for this author:
> https://en.wikipedia.org/wiki/Bohuslav_Martin%C5%AF
> 
> It seems we map this:
> https://software.hixie.ch/utilities/cgi/unicode-decoder/character-
> identifier?characters=u%CC%8A
> 
> But not:
> https://software.hixie.ch/utilities/cgi/unicode-decoder/character-
> identifier?characters=%C5%AF

Good catch Nick, I'll add a patch soon
Comment 2 Josef Moravec 2019-04-29 12:34:35 UTC
Created attachment 89010 [details] [review]
Bug 22787: Add mapping of letters "ů" and "Ů" to zebra definitions

Test plan:
0) Do not apply the patch
1) Add some word with character "ů" into metadata, for example author "Martinů, Bohuslav"
2) Try to search it with "Martinu" and you'll see you can't find it
3) Apply the patch
4) Copy file etc/zebradb/etc/word-phrase-utf.chr to your /etc/koha directory
    sudo cp etc/zebradb/etc/word-phrase-utf.chr /etc/koha/zebradb/etc/
5) koha-zebra --restart kohadev
6) koha-rebuild-zebra -f kohadev
6) try to search "Martinu" again - you should be able to find your record

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 3 Michal Denar 2019-05-02 18:49:44 UTC
Created attachment 89257 [details] [review]
Bug 22787: Add mapping of letters "ů" and "Ů" to zebra definitions

Test plan:
0) Do not apply the patch
1) Add some word with character "ů" into metadata, for example author "Martinů, Bohuslav"
2) Try to search it with "Martinu" and you'll see you can't find it
3) Apply the patch
4) Copy file etc/zebradb/etc/word-phrase-utf.chr to your /etc/koha directory
    sudo cp etc/zebradb/etc/word-phrase-utf.chr /etc/koha/zebradb/etc/
5) koha-zebra --restart kohadev
6) koha-rebuild-zebra -f kohadev
6) try to search "Martinu" again - you should be able to find your record

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Michal Denar <black23@gmail.com>
Comment 4 Marcel de Rooy 2019-05-03 09:28:57 UTC
Trying to apply it:

Traceback (most recent call last):
  File "/usr/local/bin/git-bz", line 2671, in <module>
    do_apply(*args)
  File "/usr/local/bin/git-bz", line 1747, in do_apply
    print "Bug %d - %s" % (bug.id, bug.short_desc)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u016f' in position 32: ordinal not in range(128)
Comment 5 Nick Clemens (kidclamp) 2019-05-03 12:39:29 UTC
(In reply to Marcel de Rooy from comment #4)
> Trying to apply it:
> 
> Traceback (most recent call last):
>   File "/usr/local/bin/git-bz", line 2671, in <module>
>     do_apply(*args)
>   File "/usr/local/bin/git-bz", line 1747, in do_apply
>     print "Bug %d - %s" % (bug.id, bug.short_desc)
> UnicodeEncodeError: 'ascii' codec can't encode character u'\u016f' in
> position 32: ordinal not in range(128)

Git bz error, just wget and manually apply
Comment 6 Chris Cormack 2019-05-04 07:51:56 UTC
Created attachment 89350 [details] [review]
Bug 22787: Add mapping of letters "ů" and "Ů" to zebra definitions

Test plan:
0) Do not apply the patch
1) Add some word with character "ů" into metadata, for example author "Martinů, Bohuslav"
2) Try to search it with "Martinu" and you'll see you can't find it
3) Apply the patch
4) Copy file etc/zebradb/etc/word-phrase-utf.chr to your /etc/koha directory
    sudo cp etc/zebradb/etc/word-phrase-utf.chr /etc/koha/zebradb/etc/
5) koha-zebra --restart kohadev
6) koha-rebuild-zebra -f kohadev
6) try to search "Martinu" again - you should be able to find your record

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Comment 7 Nick Clemens (kidclamp) 2019-05-07 11:45:47 UTC
Awesome work all!

Pushed to master for 19.05
Comment 8 Martin Renvoize (ashimema) 2019-05-08 15:42:50 UTC
Pushed to 18.11.x for 18.11.06
Comment 9 Lucas Gass (lukeg) 2019-05-30 14:21:54 UTC
backported to 18.05.x for 18.05.13