Bug 39485 - "Wide character in print" when exporting from staff interface and OPAC
Summary: "Wide character in print" when exporting from staff interface and OPAC
Status: Pushed to main
Alias: None
Product: Koha
Classification: Unclassified
Component: Architecture, internals, and plumbing (show other bugs)
Version: Main
Hardware: All All
: P5 - low minor
Assignee: Marcel de Rooy
QA Contact: David Cook
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2025-03-31 08:32 UTC by Marcel de Rooy
Modified: 2025-04-23 07:43 UTC (History)
1 user (show)

See Also:
GIT URL:
Change sponsored?: ---
Patch complexity: Trivial patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
25.05.00
Circulation function:


Attachments
Bug 39485: Fix wide character in print-warn in opac-export (1.77 KB, patch)
2025-03-31 08:39 UTC, Marcel de Rooy
Details | Diff | Splinter Review
Bug 39485: Fix wide character in print-warn in opac-export (1.88 KB, patch)
2025-03-31 15:54 UTC, Magnus Enger
Details | Diff | Splinter Review
Bug 39485: Fix wide character in print-warn in opac-export (1.93 KB, patch)
2025-04-22 02:36 UTC, David Cook
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Marcel de Rooy 2025-03-31 08:32:59 UTC
From a 23.11 log (lineno different)
[2025/03/16 21:11:18] [WARN] Wide character in print at /usr/share/koha/opac/opac-export.pl line 189.
Comment 1 Marcel de Rooy 2025-03-31 08:39:16 UTC Comment hidden (obsolete)
Comment 2 Magnus Enger 2025-03-31 15:54:18 UTC
Created attachment 180078 [details] [review]
Bug 39485: Fix wide character in print-warn in opac-export

Similar change in intranet counterpart: catalogue/export.

Test plan:
Verify no warning anymore when using ris or isbd when record
contains specific Unicode character(s).

Signed-off-by: Magnus Enger <magnus@libriotech.no>
I could not trigger "Wide character" for RIS in the OPAC, but for
ISBD I can see it removes that warning.
Comment 3 David Cook 2025-04-22 00:38:25 UTC
I'll look at QAing this shortly. At a glance, this change makes a lot of sense, but I'll double-check everything.
Comment 4 David Cook 2025-04-22 02:21:14 UTC
Testing on main branch in ktd with a payload of "Türkiye 我爱你" in the MARC 100$a field *before applying the patch*:

OPAC:
I get the "Wide characters in print" only for "ISBD"
[2025/04/22 02:06:56] [WARN] Wide character in print at /kohadevbox/koha/opac/opac-export.pl line 191.

Staff:
I see it for "BIBTEXT", "all Dublin core options", "MODS"
[2025/04/22 02:09:40] [WARN] Wide character in print at /kohadevbox/koha/catalogue/export.pl line 80.

NOTE: Even when I saw "Wide characters in print", the output itself seemed good. 

--

Same test *after applying the patch* and restarting Koha plack workers.

OPAC:
Can't generate the error. All the output looks good except MARC-8 (but that appears to be unrelated to this change and I think just has to do with MARC-8 not being great with Chinese?).

Staff:
Can't generate the error. All the output looks good except MARC-8 (but that appears to be unrelated to this change and I think just has to do with MARC-8 not being great with Chinese?).
Comment 5 David Cook 2025-04-22 02:35:24 UTC
The existing export code invokes some "interesting" functions from C4/Record.pm and other places...

I have a Koha plugin where I export ISO MARC and the other day I found I had to run C4::Charset::SetUTF8Flag( $marc, 1 ); on the MARC::Record object before using MARC::Record->as_usmarc even though I was printing to a filehandle that I'd opened using ":encoding(UTF-8)", so that was weird. Without that function call, it wasn't exporting the text "Türkiye" correctly. I think that's just a quirk of MARC::File::USMARC though. 

--

Anyway... this code removes the warning and all the output looks good, so I'm going to pass it. 

In theory, "binmode STDOUT, ':encoding(UTF-8)'" should be enough to tell Perl that we're outputting UTF-8 encoded bytes.
Comment 6 David Cook 2025-04-22 02:36:17 UTC
Created attachment 181236 [details] [review]
Bug 39485: Fix wide character in print-warn in opac-export

Similar change in intranet counterpart: catalogue/export.

Test plan:
Verify no warning anymore when using ris or isbd when record
contains specific Unicode character(s).

Signed-off-by: Magnus Enger <magnus@libriotech.no>
I could not trigger "Wide character" for RIS in the OPAC, but for
ISBD I can see it removes that warning.
Signed-off-by: David Cook <dcook@prosentient.com.au>
Comment 7 Katrin Fischer 2025-04-23 07:43:17 UTC
Pushed for 25.05!

Well done everyone, thank you!