---- Reported by email@example.com 2008-10-21 08:17:15 ----
Per the patch submitted for http://bugs.koha.org/cgi-bin/bugzilla3/show_bug.cgi?id=2627, the patron import now requires the input file to use UTF-8.
If non-UTF-8 characters are present, the tool does not warn about it. Instead, when the affected field is loaded, MySQL will silently truncate the contents starting with the first invalid character.
The tool should check its input. If any fields have invalid 8-bit characters, it should provide a warning and refuse to load the record.
---- Additional Comments From firstname.lastname@example.org 2009-05-06 17:21:12 ----
I'm not sure how to approach this. We only access the attachment via Text::CSV in binary mode.
Do you want a prepass for invalid characters? Or a per-line check with Encode::Guess?
We may conclude that no additional code is warranted, since the errors for Text::CSV parsing are already well reported. If the attachment is ASCII, for example, then there is no problem. What other encodings are we expecting?
It seems like no CSV editor is going to insert characters outside UTF-8 too easily. Can you attach an example CSV file with data you think should be caught by the script?
--- Bug imported by email@example.com 2010-05-21 00:54 UTC ---
This bug was previously known as _bug_ 2705 at http://bugs.koha.org/cgi-bin/bugzilla3/show_bug.cgi?id=2705
Actual time not defined. Setting to 0.0
An example where I have seen this would be a file with German umlauts in a non-UTF-8 encoding. The surname Müller would turn into M after import.
what is the dif. of UTF-8,UTF-16 and ANSI?