Bug 22787

Summary: Mapping missing for ů to u in word-phrase-utf-chr
Product: Koha Reporter: Nick Clemens <nick>
Component: SearchingAssignee: Josef Moravec <josef.moravec>
Status: CLOSED FIXED QA Contact: Testopia <testopia>
Severity: normal    
Priority: P5 - low CC: black23, chris, josef.moravec, katrin.fischer, lucas, m.de.rooy, martin.renvoize
Version: Main   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
19.05.00, 18.11.06
Attachments: Bug 22787: Add mapping of letters "ů" and "Ů" to zebra definitions
Bug 22787: Add mapping of letters "ů" and "Ů" to zebra definitions
Bug 22787: Add mapping of letters "ů" and "Ů" to zebra definitions

Comment 1 Josef Moravec 2019-04-29 12:16:14 UTC
(In reply to Nick Clemens from comment #0)
> The problem was noticed when searching for this author:
> https://en.wikipedia.org/wiki/Bohuslav_Martin%C5%AF
> 
> It seems we map this:
> https://software.hixie.ch/utilities/cgi/unicode-decoder/character-
> identifier?characters=u%CC%8A
> 
> But not:
> https://software.hixie.ch/utilities/cgi/unicode-decoder/character-
> identifier?characters=%C5%AF

Good catch Nick, I'll add a patch soon
Comment 2 Josef Moravec 2019-04-29 12:34:35 UTC
Created attachment 89010 [details] [review]
Bug 22787: Add mapping of letters "ů" and "Ů" to zebra definitions

Test plan:
0) Do not apply the patch
1) Add some word with character "ů" into metadata, for example author "Martinů, Bohuslav"
2) Try to search it with "Martinu" and you'll see you can't find it
3) Apply the patch
4) Copy file etc/zebradb/etc/word-phrase-utf.chr to your /etc/koha directory
    sudo cp etc/zebradb/etc/word-phrase-utf.chr /etc/koha/zebradb/etc/
5) koha-zebra --restart kohadev
6) koha-rebuild-zebra -f kohadev
6) try to search "Martinu" again - you should be able to find your record

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 3 Michal Denar 2019-05-02 18:49:44 UTC
Created attachment 89257 [details] [review]
Bug 22787: Add mapping of letters "ů" and "Ů" to zebra definitions

Test plan:
0) Do not apply the patch
1) Add some word with character "ů" into metadata, for example author "Martinů, Bohuslav"
2) Try to search it with "Martinu" and you'll see you can't find it
3) Apply the patch
4) Copy file etc/zebradb/etc/word-phrase-utf.chr to your /etc/koha directory
    sudo cp etc/zebradb/etc/word-phrase-utf.chr /etc/koha/zebradb/etc/
5) koha-zebra --restart kohadev
6) koha-rebuild-zebra -f kohadev
6) try to search "Martinu" again - you should be able to find your record

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Michal Denar <black23@gmail.com>
Comment 4 Marcel de Rooy 2019-05-03 09:28:57 UTC
Trying to apply it:

Traceback (most recent call last):
  File "/usr/local/bin/git-bz", line 2671, in <module>
    do_apply(*args)
  File "/usr/local/bin/git-bz", line 1747, in do_apply
    print "Bug %d - %s" % (bug.id, bug.short_desc)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u016f' in position 32: ordinal not in range(128)
Comment 5 Nick Clemens 2019-05-03 12:39:29 UTC
(In reply to Marcel de Rooy from comment #4)
> Trying to apply it:
> 
> Traceback (most recent call last):
>   File "/usr/local/bin/git-bz", line 2671, in <module>
>     do_apply(*args)
>   File "/usr/local/bin/git-bz", line 1747, in do_apply
>     print "Bug %d - %s" % (bug.id, bug.short_desc)
> UnicodeEncodeError: 'ascii' codec can't encode character u'\u016f' in
> position 32: ordinal not in range(128)

Git bz error, just wget and manually apply
Comment 6 Chris Cormack 2019-05-04 07:51:56 UTC
Created attachment 89350 [details] [review]
Bug 22787: Add mapping of letters "ů" and "Ů" to zebra definitions

Test plan:
0) Do not apply the patch
1) Add some word with character "ů" into metadata, for example author "Martinů, Bohuslav"
2) Try to search it with "Martinu" and you'll see you can't find it
3) Apply the patch
4) Copy file etc/zebradb/etc/word-phrase-utf.chr to your /etc/koha directory
    sudo cp etc/zebradb/etc/word-phrase-utf.chr /etc/koha/zebradb/etc/
5) koha-zebra --restart kohadev
6) koha-rebuild-zebra -f kohadev
6) try to search "Martinu" again - you should be able to find your record

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Comment 7 Nick Clemens 2019-05-07 11:45:47 UTC
Awesome work all!

Pushed to master for 19.05
Comment 8 Martin Renvoize 2019-05-08 15:42:50 UTC
Pushed to 18.11.x for 18.11.06
Comment 9 Lucas Gass 2019-05-30 14:21:54 UTC
backported to 18.05.x for 18.05.13