Bug 4828 - Erroneous SIP2 behaviour with non ascii characters
Summary: Erroneous SIP2 behaviour with non ascii characters
Status: CLOSED DUPLICATE of bug 6765
Alias: None
Product: Koha
Classification: Unclassified
Component: SIP2 (show other bugs)
Version: unspecified
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Ian Walls
QA Contact: Bugs List
Depends on:
Reported: 2010-05-27 14:56 UTC by Colin Campbell
Modified: 2012-10-26 00:41 UTC (History)
3 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Who signed the patch off:
Text to go in the release notes:
Version(s) released in:

Proposed fix (2.60 KB, patch)
2011-02-17 17:45 UTC, Ian Walls
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Colin Campbell 2010-05-27 14:56:56 UTC
The underlying bug is as much in the sip2 specification and implementation as in Koha,
Symptom text in a sip request or response with a non-ascii character causes "undefined" behaviour in the client. (Usually an error and reset)

Cause: The standard specifies that the sip character set is defined by an old dos codepage. I know of no client implementation which adheres to this most use a windows characterset. In Koha we send the text as retrieved from the database (i.e. utf-8) this is ok if the client is utf-8 or data is purely anglophone lacking special characters and diacritics. For some client devices we need a mechanism to protect them from scary non-ascii characters or to present them in a form they can display.

In general the suppliers of client devices are a bit reticent about their machines character set limitations. This enhancement will probably be addressed when we have a case who want it fixed. This bug has been reported before but was closed without this specific problem being addressed
Comment 1 Liz Rea 2010-10-24 21:51:25 UTC
NEKLS is having this problem now, and a RFP is probably forthcoming to address it.
Comment 2 Ian Walls 2011-02-17 17:45:41 UTC
Created attachment 3175 [details] [review]
Proposed fix
Comment 3 Dobrica Pavlinusic 2011-02-17 18:36:43 UTC
we are using just

$msg = encode('utf-8', $msg);

together with

local $/ = "\x0d";

to send out data correctly to 3M stack which is some Java stuff installed on Windows. This two changes allowed us to see accented characters correctly on self-checks, and it's developed using 3M SIP 2.00 Emulator Version

I would suggest to add configuration option which can have default to clean output but also support sending correct utf-8 to clients which support it.
Comment 5 Ian Walls 2011-08-26 16:34:41 UTC

*** This bug has been marked as a duplicate of bug 6765 ***