Bug 13801

Summary: SVC bib bad encoding
Product: Koha Reporter: Josef Moravec <josef.moravec>
Component: Web servicesAssignee: Jonathan Druart <jonathan.druart>
Status: CLOSED FIXED QA Contact: Testopia <testopia>
Severity: normal    
Priority: P5 - low CC: black23, chris, dpavlin, fridolin.somers, jonathan.druart, katrin.fischer, mtj, tomascohen
Version: Main   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: Trivial patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Attachments: Bug 13801: svc/bib - Set charset to utf-8 on printing header
Bug 13801: svc/bib - Set charset to utf-8 on printing header
[PASSED QA] Bug 13801: svc/bib - Set charset to utf-8 on printing header

Description Josef Moravec 2015-03-06 07:51:58 UTC
If you want to get marcxml bibliographic record through /svc/bib service, it return national characters badly (possibly double?) encoded. Like this:

<datafield tag="653" ind1="0" ind2=" ">
<subfield code="a">historické romány</subfield>
<subfield code="a">Ježíš Kristus</subfield>
<subfield code="a">Jeruzalém</subfield>
<subfield code="a">životopis</subfield>
</datafield>

It should be:
<datafield tag="653" ind1="0" ind2=" ">
<subfield code="a">historické romány</subfield>
<subfield code="a">Ježíš Kristus</subfield>
<subfield code="a">Jeruzalém</subfield>
<subfield code="a">životopis</subfield>
</datafield>
Comment 1 Dobrica Pavlinusic 2015-03-06 08:49:04 UTC
I can't seem to reproduce this bug on current master. Do you see that on all or just some records? Do you see it on all fields?
Comment 2 Michal Denar 2015-03-06 09:02:38 UTC
Hello,
problem is in all tested records and fields. I can give You access into our testing server, if it help. 
Complete example: http://pastebin.com/n6Fg0Ase

Thanks

Mike
Comment 3 Josef Moravec 2015-03-06 09:16:18 UTC
I've just tried the the master version from git now, on another server (new clean virtual with debian wheezy). I've imported this record from library of congress: http://lccn.loc.gov/59046328

and then tried to access it through http://<staff-interface>/cgi-bin/koha/svc/bib/1

accessed through browser (Firefox and Chromium): http://pastebin.com/ueb58YKZ

downloaded using curl: http://pastebin.com/LG09JjeB

through browser, the encoding seems bad, through curl everything is ok...

could be some problem with http headers? (just guess)
Comment 4 Jonathan Druart 2015-03-06 09:49:58 UTC
I reproduce the issue.
Comment 5 Jonathan Druart 2015-03-06 09:57:32 UTC Comment hidden (obsolete)
Comment 6 Josef Moravec 2015-03-06 10:30:46 UTC Comment hidden (obsolete)
Comment 7 Josef Moravec 2015-03-06 10:32:15 UTC
Works as expected, thank you Jonathan.
Comment 8 Katrin Fischer 2015-03-15 23:06:54 UTC
Created attachment 36906 [details] [review]
[PASSED QA] Bug 13801: svc/bib - Set charset to utf-8 on printing header

Test plan:
0/ Add unicode characters on the record biblionumber=X
1/ go on http://<staff-interface>/cgi-bin/koha/svc/bib/X
2/ Confirm you have encoding issues
3/ Apply the patch
4/ Repeat 2
5/ Confirm that all is correctly encoded

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 9 Tomás Cohen Arazi 2015-03-25 14:33:37 UTC
Patch pushed to master.

Thanks Jonathan!
Comment 10 Chris Cormack 2015-03-29 06:39:10 UTC
Pushed to 3.18.x will be in 3.18.6
Comment 11 Fridolin Somers 2015-04-22 06:53:17 UTC
Was pushed in 3.16.09
Comment 12 Fridolin Somers 2015-04-22 07:17:54 UTC
Pushed to 3.14, will be in 3.14.15.
Comment 13 Mason James 2015-04-23 03:42:28 UTC
(In reply to Fridolin SOMERS from comment #11)
> Was pushed in 3.16.09

oops, thanks for spotting that Fridolin :0)