Bug 4828

Summary: Erroneous SIP2 behaviour with non ascii characters
Product: Koha Reporter: Colin Campbell <colin.campbell>
Component: SIP2Assignee: Ian Walls <koha.sekjal>
Status: CLOSED DUPLICATE QA Contact: Bugs List <koha-bugs>
Severity: enhancement    
Priority: P5 - low CC: dpavlin, jwagner, wizzyrea
Version: unspecified   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Attachments: Proposed fix

Description Colin Campbell 2010-05-27 14:56:56 UTC
The underlying bug is as much in the sip2 specification and implementation as in Koha,
Symptom text in a sip request or response with a non-ascii character causes "undefined" behaviour in the client. (Usually an error and reset)

Cause: The standard specifies that the sip character set is defined by an old dos codepage. I know of no client implementation which adheres to this most use a windows characterset. In Koha we send the text as retrieved from the database (i.e. utf-8) this is ok if the client is utf-8 or data is purely anglophone lacking special characters and diacritics. For some client devices we need a mechanism to protect them from scary non-ascii characters or to present them in a form they can display.

In general the suppliers of client devices are a bit reticent about their machines character set limitations. This enhancement will probably be addressed when we have a case who want it fixed. This bug has been reported before but was closed without this specific problem being addressed
Comment 1 Liz Rea 2010-10-24 21:51:25 UTC
NEKLS is having this problem now, and a RFP is probably forthcoming to address it.
Comment 2 Ian Walls 2011-02-17 17:45:41 UTC
Created attachment 3175 [details] [review]
Proposed fix
Comment 3 Dobrica Pavlinusic 2011-02-17 18:36:43 UTC
we are using just

$msg = encode('utf-8', $msg);

together with

local $/ = "\x0d";

to send out data correctly to 3M stack which is some Java stuff installed on Windows. This two changes allowed us to see accented characters correctly on self-checks, and it's developed using 3M SIP 2.00 Emulator Version 1.3.0.1

I would suggest to add configuration option which can have default to clean output but also support sending correct utf-8 to clients which support it.
Comment 5 Ian Walls 2011-08-26 16:34:41 UTC

*** This bug has been marked as a duplicate of bug 6765 ***