Bug 6267

Summary: custom http user-agent in check-url.pl (fix for books.google.com 401 error)
Product: Koha Reporter: Judit <judit>
Component: Command-line UtilitiesAssignee: Galen Charlton <gmcharlt>
Status: CLOSED FIXED QA Contact: Marcel de Rooy <m.de.rooy>
Severity: trivial    
Priority: P5 - low CC: chris, f.demians, jonathan.druart, m.de.rooy, mjr, paul.poulain
Version: 3.6   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Bug Depends on:    
Bug Blocks: 7963    
Attachments: Bug 6267 – custom http user-agent in check-url.pl (fix for books.google.com 401 error)
Bug 6267 custom http user-agent in check-url.pl (fix for books.google.com 401 error)
Fix a typo
Bug 6267: [SIGNED-OFF]Fix a typo

Description Judit 2011-04-26 01:43:24 UTC
After running check-url.pl on couple of Koha instances i noticed that every single URL from books.google.com came back with 401 Unauthorized error, even though the URLs were very much alive.

I have changed the http user-agent, and now it works OK.

Here is my fix:

koha/koha.git/misc/cronjobs# diff check-url.pl check-url-calyx.pl 
88c88
<     my ($class, $timeout) = @_;
---
>     my ($class, $timeout, $agent) = @_;
91a92
>     $uagent->agent( $agent) if $agent;
159a161
> my $agent     = '';
166a169
>     'agent=s'    => \$agent,
187c190
<     my $checker = C4::URL::Checker->new($timeout);
---
>     my $checker = C4::URL::Checker->new($timeout,$agent);
246c249
< =item check-url.pl [--verbose|--help] [--host=http://default.tld] 
---
> =item check-url.pl [--verbose|--help|--agent] [--host=http://default.tld] 
266a270,275
> =item B<--agent>
> 
> Change default libwww user-agent string to custom.
> Some sites does not like libwww user agent and so check-url gives
> error even if the site works from browser.
>  

Please note that i am not a perl developer. The patch has been tested on several Koha installs, but there might be a nicer solution out there.

Sponsored by CALYX information essentials.
Comment 1 MJ Ray (software.coop) 2012-04-05 12:18:43 UTC Comment hidden (obsolete)
Comment 2 Chris Cormack 2012-04-05 21:23:16 UTC Comment hidden (obsolete)
Comment 3 Marcel de Rooy 2012-04-16 07:25:04 UTC
QA Comment: Looks good to me. Did not test it. Patch only touches a cron job, no template changes or translation issues.
Passed QA.
Comment 4 Frédéric Demians 2012-05-25 06:09:19 UTC
Due to this patch, check-url.pl script fails with this error:

syntax error at ./check-url.pl line 168, near "$agent;"
syntax error at ./check-url.pl line 170, near ")"
Execution of ./check-url.pl aborted due to compilation errors (#1)

See also alternative check URL script: bug 7963.
Comment 5 Frédéric Demians 2012-05-25 06:11:37 UTC Comment hidden (obsolete)
Comment 6 Jonathan Druart 2012-05-29 14:25:25 UTC
Created attachment 9833 [details] [review]
Bug 6267: [SIGNED-OFF]Fix a typo

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Comment 7 Paul Poulain 2012-06-09 12:38:20 UTC
QA comment : tiny & obvious patch replacing a ; by a , to avoid an error
Comment 8 Paul Poulain 2012-06-09 12:45:14 UTC
Patch pushed on master branch, I can't push it on new/bug_6267

Should be backported in 3.8
Comment 9 Chris Cormack 2012-06-10 09:39:43 UTC
Pushed to 3.8.x will be in 3.8.2