Bug 32090

Summary: Emojis (or other utf8 characters) can break background job processing
Product: Koha Reporter: Nick Clemens (kidclamp) <nick>
Component: Architecture, internals, and plumbingAssignee: Bugs List <koha-bugs>
Status: NEW --- QA Contact: Testopia <testopia>
Severity: normal    
Priority: P5 - low CC: kyle, m.de.rooy, tomascohen
Version: Main   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:

Description Nick Clemens (kidclamp) 2022-11-03 14:18:05 UTC
Change your user's first name to be:
Developer ﷐[U+1F63A]﷑

Do something that enqueues a background job

Check the logs:
Wide character in subroutine entry at /usr/share/koha/lib/Koha/BackgroundJob.pm line 170.

Comes from:
263 Returns a JSON object with utf8 disabled. Encoding to UTF-8 should be
264 done later.
265 
266 =cut
267 
268 sub json {
269     my ( $self ) = @_;
270     $self->{_json} //= JSON->new->utf8(0); # TODO Should we allow_nonref ?
271     return $self->{_json};
272 }


Maybe we should not do that later?
Comment 1 Marcel de Rooy 2022-11-03 14:39:40 UTC
(In reply to Nick Clemens from comment #0)
> Change your user's first name to be:
> Developer ﷐[U+1F63A]﷑

Ok. You are asking for trouble :)

> Maybe we should not do that later?

No. You create a lot of problems when you encode in the middle of the process. The idea is: decode at start, encode at the end. Keep everything in perl internal format in between.
Comment 2 Marcel de Rooy 2022-11-03 14:42:48 UTC
(In reply to Marcel de Rooy from comment #1)

> No. You create a lot of problems when you encode in the middle of the
> process. The idea is: decode at start, encode at the end. Keep everything in
> perl internal format in between.

Note that I had utf8 here more in mind than JSON btw. But it follows a similar pattern..