Bug 8426

Summary: Map ︡a to a and t︠ to t for searching (Non-ICU)
Product: Koha Reporter: Kyle M Hall <kyle.m.hall>
Component: Searching - ZebraAssignee: Kyle M Hall <kyle>
Status: CLOSED FIXED QA Contact:
Severity: normal    
Priority: P5 - low CC: andrewfh, fridolin.somers, jonathan.druart, katrin.fischer, martin.renvoize, nick, oleonard, paul.poulain, victor
Version: master   
Hardware: All   
OS: All   
See Also: http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=2629
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35621
Change sponsored?: --- Patch complexity: Trivial patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
21.05.00,20.11.04,20.05.10
Attachments: Bug 8426 - Diacritics not being mapped for searching
Example MARC file that uses these diacritics.
Bug 8426: (follow-up) Similar changes for Ž and ž
Bug 8426 - Diacritics not being mapped for searching
Bug 8426: Diacritics not being mapped for searching

Description Kyle M Hall 2012-07-12 11:06:53 UTC
Most of the uniform titles for Stravinsky’s music (and other Russian
composers) is in Russian.  So the uniform title for the ballet music, Firebird,
is Zhar-ptit︠s︡a which is actually two diacritics (left and right ligatures
combined). Right now it’s not searchable unless you include the ligatures in
the search term.
Comment 1 Kyle M Hall 2012-07-12 11:08:27 UTC
Created attachment 10783 [details] [review]
Bug 8426 - Diacritics not being mapped for searching

Add mappings for ︡a to a, and for t︠ to t.
Comment 2 Kyle M Hall 2012-07-12 11:09:24 UTC
Created attachment 10784 [details]
Example MARC file that uses these diacritics.
Comment 3 Katrin Fischer 2012-07-12 11:13:31 UTC
Hi Kyle, I think indexing using ICU might fix this problem. I think it could get hard to add all possible combinations to the mapping.
Comment 4 Kyle M Hall 2012-07-12 11:58:18 UTC
The library having this problem is already using ICU, and they are still experiencing this problem.
Comment 5 Jared Camins-Esakov 2012-07-12 13:49:53 UTC
(In reply to comment #4)
> The library having this problem is already using ICU, and they are still
> experiencing this problem.

If this patch has any effect, it means that Zebra has not been configured properly for ICU. Also, these ligatures are used very commonly in Russian transliteration. It's not just for ts.
Comment 6 Paul Poulain 2012-07-12 15:05:35 UTC
(In reply to comment #5)
> (In reply to comment #4)
> > The library having this problem is already using ICU, and they are still
> > experiencing this problem.
> 
> If this patch has any effect, it means that Zebra has not been configured
> properly for ICU.
I confirm : *.chr are not used in ICU mode

I think failed QA is the proper status for this patch ;-)
Comment 7 Katrin Fischer 2013-04-14 08:17:45 UTC
Kyle, is your patch still needed? I think we could probably add the mapping for libraries not using ICU, but also using ICU should fix the problem.
Comment 8 Katrin Fischer 2016-08-14 19:20:18 UTC
I have learned since that ICU doesn't support some things - so it does make sense to maintain these mappings.
Comment 9 Owen Leonard 2017-04-10 16:59:56 UTC
Is this patch still relevant? Comments here are contradictory. If this should still be tested please provide a detailed test plan.
Comment 10 Nick Clemens 2018-10-31 12:52:47 UTC Comment hidden (obsolete)
Comment 11 Nick Clemens 2018-10-31 12:56:08 UTC
(In reply to Owen Leonard from comment #9)
> Is this patch still relevant? Comments here are contradictory. If this
> should still be tested please provide a detailed test plan.

Still relevant, plan for Kyle's patch is similar to follow-up:

1 - Import the sample record
2 - Search for Zhar-ptitsa
3 - No result
4 - Apply patch
5 - You may need to cp word-phrase-utf.chr from /home/vagrant/kohaclone/etc/zebradb/etc/word-phrase-utf.chr to /etc/koha/zebradb/etc/word-phrase-utf.chr
(backup your original first)
6 - Reindex zebra
7 - Search again, you should get result
Comment 12 Nick Clemens 2019-01-07 14:30:44 UTC
(In reply to Nick Clemens from comment #11)
> (In reply to Owen Leonard from comment #9)
> > Is this patch still relevant? Comments here are contradictory. If this
> > should still be tested please provide a detailed test plan.
> 
> Still relevant, plan for Kyle's patch is similar to follow-up:
> 
> 1 - Import the sample record
> 2 - Search for Zhar-ptitsa
> 3 - No result
> 4 - Apply patch
> 5 - You may need to cp word-phrase-utf.chr from
> /home/vagrant/kohaclone/etc/zebradb/etc/word-phrase-utf.chr to
> /etc/koha/zebradb/etc/word-phrase-utf.chr
> (backup your original first)
> 6 - Reindex zebra
> 7 - Search again, you should get result

Z problems moved to bug 22073 as the patches for original issue don't seem to work
Comment 13 Martin Renvoize 2021-03-01 10:29:34 UTC
Created attachment 117434 [details] [review]
Bug 8426 - Diacritics not being mapped for searching

Add mappings for ︡a to a, and for t︠ to t.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 14 Katrin Fischer 2021-03-04 15:32:17 UTC
Created attachment 117736 [details] [review]
Bug 8426: Diacritics not being mapped for searching

Add mappings for ︡a to a, and for t︠ to t.

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 15 Katrin Fischer 2021-03-04 15:33:26 UTC
The indentation on this file is a bit problematic. The new entries use spaces, but oveall this file uses tabs. I was pondering a follow-up, but couldn't quite decide which direction I should fix.
Comment 16 Jonathan Druart 2021-03-11 15:37:37 UTC
Pushed to master for 21.05, thanks to everybody involved!
Comment 17 Fridolin Somers 2021-03-19 14:20:43 UTC
Pushed to 20.11.x for 20.11.04
Comment 18 Andrew Fuerste-Henry 2021-03-23 12:45:31 UTC
Pushed to 20.05.x for 20.05.10
Comment 19 Victor Grousset/tuxayo 2021-03-23 13:25:33 UTC
Not backported to oldoldstable (19.11.x). Feel free to ask if it's needed.