Example URL: http://some.nonexistent.tld/MāoriWomenAotearoa.pdf Job dies with "Wide character in print" Solution seems to be to encode the URL.
Created attachment 77277 [details] [review] Bug 21122: check-url-quick dies on utf8 characters in URL's To test: Make a record with a URL that has a UTF8 character, such as: http://some.nonexistent.tld/MāoriWomenAotearoa.pdf Run the check-url-quick.pl job, notice it dies at that URL Apply this patch Test again, it should work.
Created attachment 77289 [details] [review] Bug 21122: check-url-quick dies on utf8 characters in URL's To test: Make a record with a URL that has a UTF8 character, such as: http://some.nonexistent.tld/MāoriWomenAotearoa.pdf Run the check-url-quick.pl job, notice it dies at that URL Apply this patch Test again, it should work. Signed-off-by: Owen Leonard <oleonard@myacpl.org>
I got worst result without the patch: 15 http://some.nonexistent.tld/MāoriWomenAotearoa.pdf 595 No such device or address EV: error in callback (ignoring): Wide character in subroutine entry at /usr/lib/x86_64-linux-gnu/perl5/5.20/AnyEvent/Handle.pm line 1007. Wide character in print at misc/cronjobs/check-url-quick.pl line 67. 15 http://ko.wikipedia.org/wiki/위키백과:대문 596 Connection timed out EV: error in callback (ignoring): Wide character in syswrite at /usr/lib/x86_64-linux-gnu/perl5/5.20/AnyEvent/Handle.pm line 2285. EV: error in callback (ignoring): Wide character in syswrite at /usr/lib/x86_64-linux-gnu/perl5/5.20/AnyEvent/Handle.pm line 2285. EV: error in callback (ignoring): Wide character in syswrite at /usr/lib/x86_64-linux-gnu/perl5/5.20/AnyEvent/Handle.pm line 2285. EV: error in callback (ignoring): Wide character in syswrite at /usr/lib/x86_64-linux-gnu/perl5/5.20/AnyEvent/Handle.pm line 2285. [...infinite loop...]
Created attachment 77338 [details] [review] Bug 21122: check-url-quick dies on utf8 characters in URL's To test: Make a record with a URL that has a UTF8 character, such as: http://some.nonexistent.tld/MāoriWomenAotearoa.pdf Run the check-url-quick.pl job, notice it dies at that URL Apply this patch Test again, it should work. Signed-off-by: Owen Leonard <oleonard@myacpl.org> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Created attachment 77339 [details] [review] Bug 21122: Fix check-url-quick when utf8 characters in URL's To test: Make a record with a URL that has a UTF8 character, such as: http://some.nonexistent.tld/MāoriWomenAotearoa.pdf Run the check-url-quick.pl job, notice it dies at that URL Apply this patch Test again, it should work. Signed-off-by: Owen Leonard <oleonard@myacpl.org> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Pushed to master, thanks!
Pushed to 18.05.x for 18.05.03.
Pushed to 17.11.x for 17.11.09
The UTF8 URLS of the world thank you all kindly <3
I'm not quite sure if I can ask about this here, but I've been having problems when urls contain characters such as é or à. I understand that this patch makes it so check-url-quick doesn't explode when it encounters such characters, but when the http request is sent they're encoded differently and are marked as broken links (é turns into é for example). Is this related to this bug ?
(In reply to José-Mario Monteiro-Santos from comment #10) > I'm not quite sure if I can ask about this here, but I've been having > problems when urls contain characters such as é or à. I understand that this > patch makes it so check-url-quick doesn't explode when it encounters such > characters, but when the http request is sent they're encoded differently > and are marked as broken links (é turns into é for example). Is this > related to this bug ? No, or I do not understand what you meant exactly. I have tried with 856$u = https://en.wikipedia.org/wiki/Māori, then run the script with --verbose: 1 https://en.wikipedia.org/wiki/Māori 200 OK Looks ok for me.