After running check-url.pl on couple of Koha instances i noticed that every single URL from books.google.com came back with 401 Unauthorized error, even though the URLs were very much alive. I have changed the http user-agent, and now it works OK. Here is my fix: koha/koha.git/misc/cronjobs# diff check-url.pl check-url-calyx.pl 88c88 < my ($class, $timeout) = @_; --- > my ($class, $timeout, $agent) = @_; 91a92 > $uagent->agent( $agent) if $agent; 159a161 > my $agent = ''; 166a169 > 'agent=s' => \$agent, 187c190 < my $checker = C4::URL::Checker->new($timeout); --- > my $checker = C4::URL::Checker->new($timeout,$agent); 246c249 < =item check-url.pl [--verbose|--help] [--host=http://default.tld] --- > =item check-url.pl [--verbose|--help|--agent] [--host=http://default.tld] 266a270,275 > =item B<--agent> > > Change default libwww user-agent string to custom. > Some sites does not like libwww user agent and so check-url gives > error even if the site works from browser. > Please note that i am not a perl developer. The patch has been tested on several Koha installs, but there might be a nicer solution out there. Sponsored by CALYX information essentials.
Created attachment 8913 [details] [review] Bug 6267 – custom http user-agent in check-url.pl (fix for books.google.com 401 error) Patch by Judit with a small change to the help wording. Sponsored by CALYX information essentials.
Created attachment 8937 [details] [review] Bug 6267 custom http user-agent in check-url.pl (fix for books.google.com 401 error) Patch by Judit with a small change to the help wording. Sponsored by CALYX information essentials. Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
QA Comment: Looks good to me. Did not test it. Patch only touches a cron job, no template changes or translation issues. Passed QA.
Due to this patch, check-url.pl script fails with this error: syntax error at ./check-url.pl line 168, near "$agent;" syntax error at ./check-url.pl line 170, near ")" Execution of ./check-url.pl aborted due to compilation errors (#1) See also alternative check URL script: bug 7963.
Created attachment 9769 [details] [review] Fix a typo
Created attachment 9833 [details] [review] Bug 6267: [SIGNED-OFF]Fix a typo Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
QA comment : tiny & obvious patch replacing a ; by a , to avoid an error
Patch pushed on master branch, I can't push it on new/bug_6267 Should be backported in 3.8
Pushed to 3.8.x will be in 3.8.2