Conversion of MARC from ISO5426 is defined in C4::Charset::char_decode5426(). Each character or combined characters conversion is defined in a map. This conversion is missing some cases and some actual conversions are odd.
I've found only one full documentation about this conversion : http://www.gymel.com/charsets/MAB2.html Is it correct ?
Created attachment 38655 [details] [review] Bug 14078 - converting from ISO5426 is not complete Conversion of MARC from ISO5426 is defined in C4::Charset::char_decode5426(). Each character or combined characters conversion is defined in a map. This patch changes some odd actual conversions. In char_decode5426(), only characters between 0xC0 and 0xDF will be used for combining with following charater : ($char >= 0xC0 && $char <= 0xDF) So conversion like "$chars{0x81d1}=0x00b0" will never be used. Rules for "h with breve below" use combining with 0xf9 but looks like the correct caracter is 0xd5. See http://www.gymel.com/charsets/MAB2.html
Created attachment 38656 [details] [review] Bug 14078 - converting from ISO5426 is not complete - followup1 Conversion of MARC from ISO5426 is defined in C4::Charset::char_decode5426(). Each character or combined characters conversion is defined in a map. This patch adds missing conversions. See http://www.gymel.com/charsets/MAB2.html
Patches are actually drafts so I do not set to "need signoff". Looking for feedback.
Hi Fridolin, I confirm you that http://www.gymel.com/charsets/MAB2.html is a good source for ISO 5426
(In reply to Zeno Tajoli from comment #5) > Hi Fridolin, > > I confirm you that http://www.gymel.com/charsets/MAB2.html is a good source > for ISO 5426 Thanks a lot. I set has need signoff
Do you have some test data?
(In reply to M. Tompsett from comment #7) > Do you have some test data? Here are some records.
Created attachment 40534 [details] sample records
Created attachment 44051 [details] [review] Bug 14078 - converting from ISO5426 is not complete Conversion of MARC from ISO5426 is defined in C4::Charset::char_decode5426(). Each character or combined characters conversion is defined in a map. This patch changes some odd actual conversions. In char_decode5426(), only characters between 0xC0 and 0xDF will be used for combining with following charater : ($char >= 0xC0 && $char <= 0xDF) So conversion like "$chars{0x81d1}=0x00b0" will never be used. Rules for "h with breve below" use combining with 0xf9 but looks like the correct caracter is 0xd5. See http://www.gymel.com/charsets/MAB2.html Signed-off-by: Frederic Demians <f.demians@tamil.fr>
Created attachment 44052 [details] [review] Bug 14078 - converting from ISO5426 is not complete
Created attachment 44053 [details] [review] Bug 14078 - converting from ISO5426 is not complete followup1
An example contained in sample, the 38th record : Without patch : Mexique, le nouveau ¸tigre aztèque¸ ? Shannon K. O'Neil (0032-9304) With patch : Mexique, le nouveau ″tigre aztèque″ ? Shannon K. O'Neil (0032-9304)
Created attachment 44834 [details] [review] Bug 14078 - converting from ISO5426 is not complete Conversion of MARC from ISO5426 is defined in C4::Charset::char_decode5426(). Each character or combined characters conversion is defined in a map. This patch changes some odd actual conversions. In char_decode5426(), only characters between 0xC0 and 0xDF will be used for combining with following charater : ($char >= 0xC0 && $char <= 0xDF) So conversion like "$chars{0x81d1}=0x00b0" will never be used. Rules for "h with breve below" use combining with 0xf9 but looks like the correct caracter is 0xd5. See http://www.gymel.com/charsets/MAB2.html Signed-off-by: Frederic Demians <f.demians@tamil.fr> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Created attachment 44835 [details] [review] Bug 14078 - converting from ISO5426 is not complete - followup1 Conversion of MARC from ISO5426 is defined in C4::Charset::char_decode5426(). Each character or combined characters conversion is defined in a map. This patch adds missing conversions. See http://www.gymel.com/charsets/MAB2.html Signed-off-by: Frederic Demians <f.demians@tamil.fr> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Patches pushed to master. Thanks Frido!
This patch has been pushed to 3.20.x, will be in 3.20.6.
Pushed to 3.18.13, and released.