If a OPAC user enters text containing umlauts (äöü) in a patron atribute and saves it, the Staff client will get an inernal server error in members-update.pl (approving). E.g. a text like 111ä222 leads to the following in plack error log: malformed UTF-8 character in JSON string, at character offset 14 (before "\x{fffd}2","code":"T...") at (...)/Koha/Patron/Modifications.pm line 100.
Found: http://stackoverflow.com/questions/10708297/perl-convert-a-string-to-utf-8-for-json-decode
Created attachment 62538 [details] [review] Bug 18461: Internal server error while approving OPAC-edited patron attributes containing umlauts (äöü) If a OPAC user enters text containing umlauts (äöü) in a patron atribute and saves it, the Staff client will get an inernal server error in members-update.pl (approving). E.g. a text like 111ä222 leads to the following in plack error log: malformed UTF-8 character in JSON string, at character offset 14 (before "\x{fffd}2","code":"T...") at (...)/Koha/Patron/Modifications.pm line 100. To test: - Reproduce problem above - Apply patch - Verify that Home > Patron > Update records displays without error and 'New value' displays properly
Setting to 'major' because of internal server error in staff client and, if bug is triggered, you can no longer access Home > Patron > Update. (You have to send a new, umlaut-less update request from OPAC)
Created attachment 62569 [details] [review] Bug 18461: Internal server error while approving OPAC-edited patron attributes containing umlauts (äöü) If a OPAC user enters text containing umlauts (äöü) in a patron atribute and saves it, the Staff client will get an inernal server error in members-update.pl (approving). E.g. a text like 111ä222 leads to the following in plack error log: malformed UTF-8 character in JSON string, at character offset 14 (before "\x{fffd}2","code":"T...") at (...)/Koha/Patron/Modifications.pm line 100. To test: - Reproduce problem above - Apply patch - Verify that Home > Patron > Update records displays without error and 'New value' displays properly Followed test plan and it works as intended Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>
It does not work for me, I get "The passed extended_attributes is not valid JSON"
Created attachment 62617 [details] [review] Bug 18461: (bug 13757 follow-up) Do not mix decode_json and to_json Internal server error while approving OPAC-edited patron attributes containing umlauts (äöü) If a OPAC user enters text containing umlauts (äöü) in a patron atribute and saves it, the Staff client will get an inernal server error in members-update.pl (approving). E.g. a text like 111ä222 leads to the following in plack error log: malformed UTF-8 character in JSON string, at character offset 14 To test: - Reproduce problem above - Apply patch - Verify that Home > Patron > Update records displays without error and 'New value' displays properly - Approve!
Created attachment 62631 [details] [review] Bug 18461: Make tests break on wrong UTF8 data handilng This patch makes the tests fail if extended attributes handling fails due to wrong UTF-8 data handling. To test: - Run: $ sudo koha-shell kohadev k$ cd kohaclone k$ prove t/db_dependent/Koha/Patron/Modifications.t => FAIL: Tests explode Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Created attachment 62632 [details] [review] Bug 18461: (bug 13757 follow-up) Do not mix decode_json and to_json Internal server error while approving OPAC-edited patron attributes containing umlauts (äöü) If a OPAC user enters text containing umlauts (äöü) in a patron atribute and saves it, the Staff client will get an inernal server error in members-update.pl (approving). E.g. a text like 111ä222 leads to the following in plack error log: malformed UTF-8 character in JSON string, at character offset 14 To test: - Reproduce problem above - Apply patch - Verify that Home > Patron > Update records displays without error and 'New value' displays properly - Approve! Also, run the tests: - Run: $ sudo koha-shell kohadev k$ cd kohaclone k$ prove t/db_dependent/Koha/Patron/Modifications.t => FAIL: Tests explode due to encoding problems in JSON handling - Apply the patch - Run the tests again: k$ prove t/db_dependent/Koha/Patron/Modifications.t => SUCCESS: Tests pass! Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Passing QA here but cant attach patches, ironically, because of the umlauts in the commit messages: UnicodeEncodeError: 'ascii' codec can't encode characters in position 116-118: ordinal not in range(128)
(In reply to Nick Clemens from comment #9) > Passing QA here but cant attach patches, ironically, because of the umlauts > in the commit messages: > UnicodeEncodeError: 'ascii' codec can't encode characters in position > 116-118: ordinal not in range(128) Should work without the -e option of git-bz
Created attachment 62666 [details] [review] Bug 18461: Make tests break on wrong UTF8 data handilng This patch makes the tests fail if extended attributes handling fails due to wrong UTF-8 data handling. To test: - Run: $ sudo koha-shell kohadev k$ cd kohaclone k$ prove t/db_dependent/Koha/Patron/Modifications.t => FAIL: Tests explode Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Created attachment 62667 [details] [review] Bug 18461: (bug 13757 follow-up) Do not mix decode_json and to_json Internal server error while approving OPAC-edited patron attributes containing umlauts (äöü) If a OPAC user enters text containing umlauts (äöü) in a patron atribute and saves it, the Staff client will get an inernal server error in members-update.pl (approving). E.g. a text like 111ä222 leads to the following in plack error log: malformed UTF-8 character in JSON string, at character offset 14 To test: - Reproduce problem above - Apply patch - Verify that Home > Patron > Update records displays without error and 'New value' displays properly - Approve! Also, run the tests: - Run: $ sudo koha-shell kohadev k$ cd kohaclone k$ prove t/db_dependent/Koha/Patron/Modifications.t => FAIL: Tests explode due to encoding problems in JSON handling - Apply the patch - Run the tests again: k$ prove t/db_dependent/Koha/Patron/Modifications.t => SUCCESS: Tests pass! Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Pushed to master for 17.05, thanks Tomas, Jonathan!
Depends on bug 13757 which is not part of 16.11.x.