Bug 8889

Summary: Optionally compose Unicode characters when saving MARC record
Product: Koha Reporter: Marcel de Rooy <m.de.rooy>
Component: I18N/L10NAssignee: Marcel de Rooy <m.de.rooy>
Status: CLOSED FIXED QA Contact:
Severity: enhancement    
Priority: P5 - low CC: f.demians, katrin.fischer
Version: Main   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:

Description Marcel de Rooy 2012-10-08 11:27:15 UTC
Conversion from MARC8 to UTF8 (in e.g. Z3950 search) converts a lot of accented characters to the Unicode decomposed form of [letter without accent][accent only] in Unicode.
Most characters originating from the Latin1 character set are not displayed properly in various browsers (in that decomposed form). When we convert to the composed form, these characters are mapped to one unicode character with accent. Presentation improves in various browsers.

This patch adds a syspref making this optional behavior. It makes the needed changes in ModBiblioMarc.
Comment 1 Jared Camins-Esakov 2012-10-08 11:47:02 UTC
I am not sure this should be optional. NFC is required by Zebra, and in theory ModZebra was already patched to force NFC a couple of years ago (though this was apparently unsuccessful). I would be comfortable with forcing NFC all the time, because anyone who disables the NFC composition would end up with a system that had both display and searching issues.
Comment 2 Marcel de Rooy 2012-10-08 11:54:17 UTC
I may have been too fast with this record. I now think that we probably only need to call SetUTF8Flag without second parameter more in one or two places. This function normalizes to NFC. Only some export scripts send out NFD.
Comment 3 Marcel de Rooy 2012-10-08 11:54:36 UTC
s/record/report
Comment 4 Katrin Fischer 2012-10-08 11:57:34 UTC
I know that there currently is a problem when downloading records from an utf8 source that gives you the decomposed form. It would be great if that could be corrected automatically.
Comment 5 Marcel de Rooy 2012-10-08 12:16:52 UTC
This problem seems to be resolved by bug 8610.