Bug 18461 - Internal server error while approving OPAC-edited patron attributes containing umlauts (äöü)
Summary: Internal server error while approving OPAC-edited patron attributes containin...
Status: RESOLVED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Patrons (show other bugs)
Version: master
Hardware: All All
: P5 - low critical (vote)
Assignee: Jonathan Druart
QA Contact: Testopia
URL:
Whiteboard:
Keywords:
Depends on: 13757
Blocks:
  Show dependency treegraph
 
Reported: 2017-04-19 20:02 UTC by Marc Véron
Modified: 2017-05-13 11:56 UTC (History)
8 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Trivial patch
Bot Control: ---
When did the bot last check this:
Who signed the patch off:
Text to go in the release notes:


Attachments
Bug 18461: Internal server error while approving OPAC-edited patron attributes containing umlauts (äöü) (1.72 KB, patch)
2017-04-21 16:26 UTC, Marc Véron
Details | Diff | Splinter Review
Bug 18461: Internal server error while approving OPAC-edited patron attributes containing umlauts (äöü) (1.82 KB, patch)
2017-04-22 12:07 UTC, Alex Buckley
Details | Diff | Splinter Review
Bug 18461: (bug 13757 follow-up) Do not mix decode_json and to_json (2.25 KB, patch)
2017-04-24 16:58 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 18461: Make tests break on wrong UTF8 data handilng (2.08 KB, patch)
2017-04-24 18:50 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
Bug 18461: (bug 13757 follow-up) Do not mix decode_json and to_json (2.61 KB, patch)
2017-04-24 18:50 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
Bug 18461: Make tests break on wrong UTF8 data handilng (2.13 KB, patch)
2017-04-25 13:10 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 18461: (bug 13757 follow-up) Do not mix decode_json and to_json (2.67 KB, patch)
2017-04-25 13:10 UTC, Nick Clemens
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Marc Véron 2017-04-19 20:02:40 UTC
If a OPAC user enters text containing umlauts (äöü) in a patron atribute and saves it, the Staff client will get an inernal server error in members-update.pl (approving).

E.g. a text like 111ä222 leads to the following in plack error log:

malformed UTF-8 character in JSON string, at character offset 14 (before "\x{fffd}2","code":"T...") at (...)/Koha/Patron/Modifications.pm line 100.
Comment 2 Marc Véron 2017-04-21 16:26:26 UTC
Created attachment 62538 [details] [review]
Bug 18461: Internal server error while approving OPAC-edited patron attributes containing umlauts (äöü)

If a OPAC user enters text containing umlauts (äöü) in a patron atribute and
saves it, the Staff client will get an inernal server error in
members-update.pl (approving).

E.g. a text like 111ä222 leads to the following in plack error log:

malformed UTF-8 character in JSON string, at character offset 14 (before
"\x{fffd}2","code":"T...") at (...)/Koha/Patron/Modifications.pm line 100.

To test:
- Reproduce problem above
- Apply patch
- Verify that Home > Patron > Update records displays without error
  and 'New value' displays properly
Comment 3 Marc Véron 2017-04-21 16:29:13 UTC
Setting to 'major' because of internal server error in staff client and,
if bug is triggered, you can no longer access Home > Patron > Update.
(You have to send a new, umlaut-less update request from OPAC)
Comment 4 Alex Buckley 2017-04-22 12:07:08 UTC
Created attachment 62569 [details] [review]
Bug 18461: Internal server error while approving OPAC-edited patron attributes containing umlauts (äöü)

If a OPAC user enters text containing umlauts (äöü) in a patron atribute and
saves it, the Staff client will get an inernal server error in
members-update.pl (approving).

E.g. a text like 111ä222 leads to the following in plack error log:

malformed UTF-8 character in JSON string, at character offset 14 (before
"\x{fffd}2","code":"T...") at (...)/Koha/Patron/Modifications.pm line 100.

To test:
- Reproduce problem above
- Apply patch
- Verify that Home > Patron > Update records displays without error
  and 'New value' displays properly

Followed test plan and it works as intended

Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>
Comment 5 Jonathan Druart 2017-04-24 16:51:36 UTC
It does not work for me, I get "The passed extended_attributes is not valid JSON"
Comment 6 Jonathan Druart 2017-04-24 16:58:13 UTC
Created attachment 62617 [details] [review]
Bug 18461: (bug 13757 follow-up) Do not mix decode_json and to_json

Internal server error while approving OPAC-edited patron attributes
containing umlauts (äöü)

If a OPAC user enters text containing umlauts (äöü) in a patron atribute
and saves it, the Staff client will get an inernal server error in
members-update.pl (approving).

E.g. a text like 111ä222 leads to the following in plack error log:

malformed UTF-8 character in JSON string, at character offset 14

To test:
- Reproduce problem above
- Apply patch
- Verify that Home > Patron > Update records displays without error
and 'New value' displays properly
- Approve!
Comment 7 Tomás Cohen Arazi 2017-04-24 18:50:45 UTC
Created attachment 62631 [details] [review]
Bug 18461: Make tests break on wrong UTF8 data handilng

This patch makes the tests fail if extended attributes handling fails due to
wrong UTF-8 data handling.

To test:
- Run:
  $ sudo koha-shell kohadev
 k$ cd kohaclone
 k$ prove t/db_dependent/Koha/Patron/Modifications.t
=> FAIL: Tests explode

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Comment 8 Tomás Cohen Arazi 2017-04-24 18:50:50 UTC
Created attachment 62632 [details] [review]
Bug 18461: (bug 13757 follow-up) Do not mix decode_json and to_json

Internal server error while approving OPAC-edited patron attributes
containing umlauts (äöü)

If a OPAC user enters text containing umlauts (äöü) in a patron atribute
and saves it, the Staff client will get an inernal server error in
members-update.pl (approving).

E.g. a text like 111ä222 leads to the following in plack error log:

malformed UTF-8 character in JSON string, at character offset 14

To test:
- Reproduce problem above
- Apply patch
- Verify that Home > Patron > Update records displays without error
and 'New value' displays properly
- Approve!

Also, run the tests:
- Run:
  $ sudo koha-shell kohadev
 k$ cd kohaclone
 k$ prove t/db_dependent/Koha/Patron/Modifications.t
=> FAIL: Tests explode due to encoding problems in JSON handling
- Apply the patch
- Run the tests again:
 k$ prove t/db_dependent/Koha/Patron/Modifications.t
=> SUCCESS: Tests pass!

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Comment 9 Nick Clemens 2017-04-25 10:02:26 UTC
Passing QA here but cant attach patches, ironically, because of the umlauts in the commit messages:
UnicodeEncodeError: 'ascii' codec can't encode characters in position 116-118: ordinal not in range(128)
Comment 10 Jonathan Druart 2017-04-25 12:33:12 UTC
(In reply to Nick Clemens from comment #9)
> Passing QA here but cant attach patches, ironically, because of the umlauts
> in the commit messages:
> UnicodeEncodeError: 'ascii' codec can't encode characters in position
> 116-118: ordinal not in range(128)

Should work without the -e option of git-bz
Comment 11 Nick Clemens 2017-04-25 13:10:48 UTC
Created attachment 62666 [details] [review]
Bug 18461: Make tests break on wrong UTF8 data handilng

This patch makes the tests fail if extended attributes handling fails due to
wrong UTF-8 data handling.

To test:
- Run:
  $ sudo koha-shell kohadev
 k$ cd kohaclone
 k$ prove t/db_dependent/Koha/Patron/Modifications.t
=> FAIL: Tests explode

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 12 Nick Clemens 2017-04-25 13:10:52 UTC
Created attachment 62667 [details] [review]
Bug 18461: (bug 13757 follow-up) Do not mix decode_json and to_json

Internal server error while approving OPAC-edited patron attributes
containing umlauts (äöü)

If a OPAC user enters text containing umlauts (äöü) in a patron atribute
and saves it, the Staff client will get an inernal server error in
members-update.pl (approving).

E.g. a text like 111ä222 leads to the following in plack error log:

malformed UTF-8 character in JSON string, at character offset 14

To test:
- Reproduce problem above
- Apply patch
- Verify that Home > Patron > Update records displays without error
and 'New value' displays properly
- Approve!

Also, run the tests:
- Run:
  $ sudo koha-shell kohadev
 k$ cd kohaclone
 k$ prove t/db_dependent/Koha/Patron/Modifications.t
=> FAIL: Tests explode due to encoding problems in JSON handling
- Apply the patch
- Run the tests again:
 k$ prove t/db_dependent/Koha/Patron/Modifications.t
=> SUCCESS: Tests pass!

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 13 Kyle M Hall 2017-04-28 10:25:28 UTC
Pushed to master for 17.05, thanks Tomas, Jonathan!
Comment 14 Katrin Fischer 2017-05-13 11:56:19 UTC
Depends on bug 13757 which is not part of 16.11.x.