Bug 6267 - custom http user-agent in check-url.pl (fix for books.google.com 401 error)
Summary: custom http user-agent in check-url.pl (fix for books.google.com 401 error)
Alias: None
Product: Koha
Classification: Unclassified
Component: Command-line Utilities (show other bugs)
Version: 3.6
Hardware: All All
: P5 - low trivial (vote)
Assignee: Galen Charlton
QA Contact: Marcel de Rooy
Depends on:
Blocks: 7963
  Show dependency treegraph
Reported: 2011-04-26 01:43 UTC by Judit
Modified: 2013-12-05 19:57 UTC (History)
6 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Who signed the patch off:
Text to go in the release notes:
Version(s) released in:

Bug 6267 – custom http user-agent in check-url.pl (fix for books.google.com 401 error) (2.61 KB, patch)
2012-04-05 12:18 UTC, MJ Ray (software.coop)
Details | Diff | Splinter Review
Bug 6267 custom http user-agent in check-url.pl (fix for books.google.com 401 error) (2.53 KB, patch)
2012-04-05 21:23 UTC, Chris Cormack
Details | Diff | Splinter Review
Fix a typo (724 bytes, patch)
2012-05-25 06:11 UTC, Frédéric Demians
Details | Diff | Splinter Review
Bug 6267: [SIGNED-OFF]Fix a typo (798 bytes, patch)
2012-05-29 14:25 UTC, Jonathan Druart
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Judit 2011-04-26 01:43:24 UTC
After running check-url.pl on couple of Koha instances i noticed that every single URL from books.google.com came back with 401 Unauthorized error, even though the URLs were very much alive.

I have changed the http user-agent, and now it works OK.

Here is my fix:

koha/koha.git/misc/cronjobs# diff check-url.pl check-url-calyx.pl 
<     my ($class, $timeout) = @_;
>     my ($class, $timeout, $agent) = @_;
>     $uagent->agent( $agent) if $agent;
> my $agent     = '';
>     'agent=s'    => \$agent,
<     my $checker = C4::URL::Checker->new($timeout);
>     my $checker = C4::URL::Checker->new($timeout,$agent);
< =item check-url.pl [--verbose|--help] [--host=http://default.tld] 
> =item check-url.pl [--verbose|--help|--agent] [--host=http://default.tld] 
> =item B<--agent>
> Change default libwww user-agent string to custom.
> Some sites does not like libwww user agent and so check-url gives
> error even if the site works from browser.

Please note that i am not a perl developer. The patch has been tested on several Koha installs, but there might be a nicer solution out there.

Sponsored by CALYX information essentials.
Comment 1 MJ Ray (software.coop) 2012-04-05 12:18:43 UTC Comment hidden (obsolete)
Comment 2 Chris Cormack 2012-04-05 21:23:16 UTC Comment hidden (obsolete)
Comment 3 Marcel de Rooy 2012-04-16 07:25:04 UTC
QA Comment: Looks good to me. Did not test it. Patch only touches a cron job, no template changes or translation issues.
Passed QA.
Comment 4 Frédéric Demians 2012-05-25 06:09:19 UTC
Due to this patch, check-url.pl script fails with this error:

syntax error at ./check-url.pl line 168, near "$agent;"
syntax error at ./check-url.pl line 170, near ")"
Execution of ./check-url.pl aborted due to compilation errors (#1)

See also alternative check URL script: bug 7963.
Comment 5 Frédéric Demians 2012-05-25 06:11:37 UTC Comment hidden (obsolete)
Comment 6 Jonathan Druart 2012-05-29 14:25:25 UTC
Created attachment 9833 [details] [review]
Bug 6267: [SIGNED-OFF]Fix a typo

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Comment 7 Paul Poulain 2012-06-09 12:38:20 UTC
QA comment : tiny & obvious patch replacing a ; by a , to avoid an error
Comment 8 Paul Poulain 2012-06-09 12:45:14 UTC
Patch pushed on master branch, I can't push it on new/bug_6267

Should be backported in 3.8
Comment 9 Chris Cormack 2012-06-10 09:39:43 UTC
Pushed to 3.8.x will be in 3.8.2