Triggered by running Circulation.t on bug 18677: t/db_dependent/Circulation.t .. 1/121 unac_string: Invalid or incomplete multibyte or wide character unac_string: Invalid or incomplete multibyte or wide character Note that Text::Unaccent says: Return the unaccented equivalent of the string $string. The character set of $string is specified by the $charset argument. The returned string is coded using the same character set. The only place we call unac_string is: Koha::Patron->generate_userid: $userid = unac_string('utf-8',$userid); Since we say to unac_string that $userid is utf-8, we should first encode it and the result we should decode again for further handling.
Created attachment 82384 [details] [review] Bug 21848: Handle encoding when calling Text::Unaccent::unac_string Resolve warnings like: unac_string: Invalid or incomplete multibyte or wide character Note that Text::Unaccent says: Return the unaccented equivalent of the string $string. The character set of $string is specified by the $charset argument. The returned string is coded using the same character set. The only place we call unac_string is: Koha::Patron->generate_userid: $userid = unac_string('utf-8',$userid); Since we say to unac_string that $userid is utf-8, we should first encode it and the result we should decode again for further handling. Test plan: [1] Run t/db_dependent/Koha/Patrons.t [2] Run t/db_dependent/Circulation.t (before this patch it gave a few warns) Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Created attachment 82385 [details] [review] Bug 21848: Remove Text::Unaccent from C4::Members At refactoring time the unac_string call was moved to Koha::Patron. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signing off in stream of 18677
Tomas: Please QA. It is all about tomasito ;)
Marcel, do you understand why we do not get the warning from Patrons.t?
I think that we should add a test to show that we removed the warning.
(In reply to Jonathan Druart from comment #5) > Marcel, do you understand why we do not get the warning from Patrons.t? Since we do not use special chars in the names there. Which we did in Circulation.t. At least I see: my %expired_borrower_data = ( firstname => 'Ça', surname => 'Glisse', categorycode => $patron_category->{categorycode}, branchcode => $branch, dateexpiry => dt_from_string->subtract( months => 1 ), ); […] Koha::Patron->new(\%expired_borrower_data)->store->borrowernumber; Yeah, renaming Ça is enough to get rid of the warn.
Hmm. Something strange going on here. Note that adding use utf8; to Circulation.t also resolves the warning. Having trouble to catch the warn with warning_like and its friends. Should we just add "use utf8;" and leave Patron.pm alone ?
Created attachment 83146 [details] [review] Bug 21848: Counterpatch to resolve unac_string warning Just add "use utf8". Test plan: Prove Circulation.t without warnings. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment on attachment 82384 [details] [review] Bug 21848: Handle encoding when calling Text::Unaccent::unac_string Review of attachment 82384 [details] [review]: ----------------------------------------------------------------- ::: Koha/Patron.pm @@ +25,3 @@ > use List::MoreUtils qw( uniq ); > use JSON qw( to_json ); > use Text::Unaccent qw( unac_string ); Do we still need to load Text::Unaccent here?
Created attachment 83210 [details] [review] Bug 21848: Resolve unac_string warning from Circulation.t Just add "use utf8". Test plan: Run Circulation.t; verify there are no warnings. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Created attachment 83211 [details] [review] Bug 21848: Remove Text::Unaccent from C4::Members At refactoring time the unac_string call was moved to Koha::Patron. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Jonathan: I decided to simplify here.
Created attachment 83217 [details] [review] Bug 21848: Resolve unac_string warning from Circulation.t Just add "use utf8". Test plan: Run Circulation.t; verify there are no warnings. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Created attachment 83218 [details] [review] Bug 21848: Remove Text::Unaccent from C4::Members At refactoring time the unac_string call was moved to Koha::Patron. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Awesome work all! Pushed to master for 19.05
Pushed to 18.11.x for 18.11.01
backported to 18.05 for 18.05.08
Pushed to 17.11.x for 17.11.14
Ah brakes Jenkins. In the end the warn does not exist in 17.11.x. Caused by Bug 19936 who moved generate_userid to Koha::Patron.