Bug 17717 - Fix broken cronjobs due to permissions of the current directory
Summary: Fix broken cronjobs due to permissions of the current directory
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Command-line Utilities (show other bugs)
Version: Main
Hardware: All Linux
: P4 critical (vote)
Assignee: Tomás Cohen Arazi
QA Contact: Marcel de Rooy
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-12-03 13:50 UTC by mjnkml
Modified: 2021-12-13 21:10 UTC (History)
21 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Small patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
Full error as displayed in the email (3.14 KB, text/plain)
2016-12-03 13:58 UTC, mjnkml
Details
Bug 17717 : Work around for the permissions problem when running process_message_queue (1.23 KB, patch)
2017-05-23 20:29 UTC, Chris Cormack
Details | Diff | Splinter Review
[SIGNED-OFF] Bug 17717 : Work around for the permissions problem when running process_message_queue (1.29 KB, patch)
2017-05-23 20:59 UTC, Josef Moravec
Details | Diff | Splinter Review
Bug 17717: Add a --chdir option switch for koha-foreach (5.29 KB, patch)
2018-03-02 15:20 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
Bug 17717: Make cronjobs using koha-foreach use --chdir (4.65 KB, patch)
2018-03-02 15:20 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
Bug 17717: Add a --chdir option switch for koha-foreach (5.33 KB, patch)
2018-03-02 15:50 UTC, Kyle M Hall
Details | Diff | Splinter Review
Bug 17717: Make cronjobs using koha-foreach use --chdir (4.69 KB, patch)
2018-03-02 15:50 UTC, Kyle M Hall
Details | Diff | Splinter Review
Bug 17717: Add a --chdir option switch for koha-foreach (5.43 KB, patch)
2018-03-05 10:36 UTC, Marcel de Rooy
Details | Diff | Splinter Review
Bug 17717: Make cronjobs using koha-foreach use --chdir (4.78 KB, patch)
2018-03-05 10:36 UTC, Marcel de Rooy
Details | Diff | Splinter Review
Bug 17717: (QA follow-up) Fix typo chdir (1.15 KB, patch)
2018-03-05 10:36 UTC, Marcel de Rooy
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description mjnkml 2016-12-03 13:50:43 UTC
On Ubuntu, output of /usr/share/koha/bin/cronjobs/process_message_queue.pl in an email from cron:

ourlib: Can't locate Authen/CAS/Client/Response/Failure.pm:   ./Authen/CAS/Client/Response/Failure.pm: Permission denied at /usr/share/perl/5.22/base.pm line 97.
ourlib:        ...propagated at /usr/share/perl/5.22/base.pm line 106.
ourlib: BEGIN failed--compilation aborted at /usr/share/perl5/Authen/CAS/Client/Response.pm line 68.
ourlib: Compilation failed in require at /usr/share/perl5/Authen/CAS/Client.pm line 8.
.
.
.

process_message_queue.pl is currently invoked from /etc/cron.d/koha-common

I don't find any Failure.pm on the system.

This only started happening after a recent update which resulted in Bug 17234. Not sure if issues with that may have also affected this. Have done a full reinstall (without Purge) of the packages.
Comment 1 mjnkml 2016-12-03 13:58:51 UTC
Created attachment 57947 [details]
Full error as displayed in the email
Comment 2 mjnkml 2016-12-03 14:08:10 UTC
After extracting Failure to its own file, process_message_queue.pl runs without a peep. I don't understand enough of Perl to know what's going on.

Files end up like this:

root@koha:/usr/share/perl5/Authen/CAS/Client# find
.
./Response.pm.bak
./Response
./Response/Failure.pm
./Response.pm

Failure.pm is cut out of Response.pm: (didn't have much success with multi-line comments!)

root@koha:/usr/share/perl5/Authen/CAS/Client# cat Response/Failure.pm
require 5.006_001;

use strict;
use warnings;

#======================================================================
# Authen::CAS::Client::Response::Failure
#
package Authen::CAS::Client::Response::Failure;

use base qw/ Authen::CAS::Client::Response /;

sub _ATTRIBUTES () { code => undef, message => '', $_[0]->SUPER::_ATTRIBUTES }

sub new { my $class = shift; $class->SUPER::new( @_, _ok => 0 ) }

sub code    { my ( $self ) = @_; $self->{code} }
sub message { my ( $self ) = @_; $self->{message} }
Comment 3 mjnkml 2016-12-05 00:47:21 UTC
After separating Failure.pm, the emails are now showing the same problem with Success.pm.
After extracting Success.pm in the same way, now it complains about Exceptions.pm :

Can't locate Koha/Exceptions/Exception.pm:   ./Koha/Exceptions/Exception.pm: Permission denied at /usr/share/perl/5.22/base.pm line 97, <DATA> line 755

Seems like a problem with the build, or (the current version of?) perl not supporting multiple classes within a .pm file?
Comment 4 mjnkml 2016-12-05 03:45:50 UTC
Futher, after separating Exception.pm, we get:

ourlib: Class Koha::Exceptions::Exception appears to be a typo as it is only specified in the 'isa' param for Koha::Exceptions::Virtualshelves::InvalidInviteKey
ourlib: BEGIN failed--compilation aborted at /usr/share/koha/lib/Koha/Exceptions.pm line 41, <DATA> line 755.
.
.

Exceptions.pm itself has many references to Exception (I didn't separate all of them).

Anyhow, I think all the above problems are just a source of one very small thing which I'm not seeing / understanding. I doubt the classes needed to be separated out...
Comment 5 mjnkml 2016-12-05 04:06:29 UTC
Separating Exception.pm seemed to have broken other things, so I've backed out of it.
Comment 6 Stefan Berndtsson 2016-12-06 14:36:27 UTC
I get pretty much the exact same error when trying a koha-rebuild-zebra, so I think this is a bit more generic than the message queue.

The error occurs in a file not supplied by Koha, and when called from a core perl file for perl 5.22. It seems 5.18 (the perl version from Ubuntu 14.04) does things differently than 5.22 does.

It all looks weird, because the CAS code from Ubuntu 14.04 is pretty much the same as the one from 16.04, so that method of using multiple modules in a single file seems to have worked better back then. I think I'll have to back out to 14.04 again for the time being, without being able to run plack.
Comment 7 Chris Cormack 2017-03-22 03:05:47 UTC
Can confirm this is happening with Ubuntu 16.04 (as the webserver db on another server) and Koha 16.11.05

No mail is sent with this bug

Can't locate Authen/CAS/Client/Response/Failure.pm:   ./Authen/CAS/Client/Response/Failure.pm: Permission denied at /usr/share/perl/5.22/base.pm line 97.
Comment 8 Liz Rea 2017-03-22 21:41:15 UTC
On affected servers, changing the /etc/cron.d/koha-common job for process message queue to the following:

*/15 * * * * root cd /tmp && koha-foreach --enabled --email /usr/share/koha/bin/cronjobs/process_message_queue.pl

seems to end run the issue.

Thanks to Chris C. for the work around.

As Chris says: 
<rangi> its a permissions issue, something in perl 5.22 has changed
<rangi> its not actually a Koha problem, there are a few perl modules that have the same problem
Comment 9 mjnkml 2017-03-25 11:08:14 UTC
(In reply to Liz Rea from comment #8)
> On affected servers, changing the /etc/cron.d/koha-common job for process
> message queue to the following:
> 
> */15 * * * * root cd /tmp && koha-foreach --enabled --email
> /usr/share/koha/bin/cronjobs/process_message_queue.pl
> 
> seems to end run the issue.
> 
> Thanks to Chris C. for the work around.
> 
> As Chris says: 
> <rangi> its a permissions issue, something in perl 5.22 has changed
> <rangi> its not actually a Koha problem, there are a few perl modules that
> have the same problem

Ah finally - a workaround!
Comment 10 Katrin Fischer 2017-04-02 16:54:47 UTC
What should we do about this?
Comment 11 Chris Cormack 2017-04-02 21:32:16 UTC
We could do a patch that adds the cd /tmp && 

Otherwise we could try to fix the upstream module, to work better with perl 5.22 and cron.

But yeah, it's an upstream problem, I think the module is pretty much unmaintained, maybe finding a better module is a better plan.

Maybe the cd /tmp && for cron patch, and then work on a long term solution?
Comment 12 Barton Chittenden 2017-04-12 01:46:07 UTC
Note: this seems to be the same compilation error that was tripping up plack in Bug 16138
Comment 13 Tomás Cohen Arazi 2017-04-13 12:16:06 UTC
http://www.perlmonks.org/?node_id=1127880
Comment 14 Larry Baerveldt 2017-04-14 19:28:51 UTC
Just a FYI on this. I found that *not* using koha-foreach, and instead use the actual koha instance user also works.

For example, instead of:
root koha-foreach --enabled --email /usr/share/koha/bin/cronjobs/process_message_queue.pl

if you use:
INST-koha /usr/share/koha/bin/cronjobs/process_message_queue.pl

it works. 

So taking a look at how koha-foreach switches to the user might provide some clues. 

Can confirm this also happens on Debian 8.4.
Comment 15 Jonathan Druart 2017-05-23 17:18:35 UTC
So, what do we do?
Do we release 17.05 as it or we propose a workaround?
Is the `cd /tmp` trick added to the cron line acceptable?
Comment 16 Josef Moravec 2017-05-23 18:44:47 UTC
(In reply to Jonathan Druart from comment #15)
> So, what do we do?
> Do we release 17.05 as it or we propose a workaround?
> Is the `cd /tmp` trick added to the cron line acceptable?

I think we should make workaround now and create new bug report for long term solution immedietely...
Comment 17 Chris Cormack 2017-05-23 20:06:15 UTC
I am working on replacing this module with one that supports CAS 2.0 and doesnt have this problem.
It won't obviously be ready in time though. So I will write a patch to add the cd /tmp && and attach it to here.

Should be ready soon
Comment 18 Chris Cormack 2017-05-23 20:17:18 UTC
(In reply to Larry Baerveldt from comment #14)
> Just a FYI on this. I found that *not* using koha-foreach, and instead use
> the actual koha instance user also works.
> 
> For example, instead of:
> root koha-foreach --enabled --email
> /usr/share/koha/bin/cronjobs/process_message_queue.pl
> 
> if you use:
> INST-koha /usr/share/koha/bin/cronjobs/process_message_queue.pl
> 
> it works. 
> 
> So taking a look at how koha-foreach switches to the user might provide some
> clues. 
> 
> Can confirm this also happens on Debian 8.4.

Yep thats because you are running the script as root then. Root has permission to write in that dir. With for-each you are running as the user, it doesn't.

However the real problem is this module. Which we need to replace, the workaround should hold for then
Comment 19 Chris Cormack 2017-05-23 20:29:10 UTC
Created attachment 63676 [details] [review]
Bug 17717 : Work around for the permissions problem when running process_message_queue

To test
1/ Make sure some things are in the message queue
2/ Run the cron jobs, notice nothing is sent
3/ Build and install a new package (or manually update the
/etc/cron.d/koha-common file)
4/ Run the jobs again, notice mail is sent
Comment 20 Josef Moravec 2017-05-23 20:59:19 UTC
Created attachment 63677 [details] [review]
[SIGNED-OFF] Bug 17717 : Work around for the permissions problem when running process_message_queue

To test
1/ Make sure some things are in the message queue
2/ Run the cron jobs, notice nothing is sent
3/ Build and install a new package (or manually update the
/etc/cron.d/koha-common file)
4/ Run the jobs again, notice mail is sent

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Comment 21 Julian Maurice 2017-05-24 15:57:11 UTC
This bug really confuses me.

(In reply to Chris Cormack from comment #18) 
> Yep thats because you are running the script as root then. Root has
> permission to write in that dir. With for-each you are running as the user,
> it doesn't.
Why does Perl need write access to the current directory for loading a module ?

> Can't locate Authen/CAS/Client/Response/Failure.pm:   ./Authen/CAS/Client/Response/Failure.pm: Permission denied at /usr/share/perl/5.22/base.pm line 97.
suggests that ./Authen/CAS/Client/Response/Failure.pm exists but is not readable. This file shouldn't exist as Authen::CAS::Client::Response::Failure is defined in Authen/CAS/Client/Response.pm. Why does it exist ?

(In reply to Josef Moravec from comment #20)
> To test
> 1/ Make sure some things are in the message queue

Why ? What could possibly happen during process_message_queue.pl that trigger an error related to a CAS module ?

And why the solution to this problem is 'cd /tmp' ? :)
Comment 22 Jonathan Druart 2017-05-29 17:55:05 UTC
(In reply to Julian Maurice from comment #21)
> This bug really confuses me.

Same here :)
Chris do you have the answers?
Comment 23 Chris Cormack 2017-05-29 19:08:48 UTC
Ok

The original report of the error describes the problem, so I don't think i need to go proving it exists?

I'll do a summary anyway. With perl 5.22 with latest ubuntu, debian testing and above, etc have. Something has changed in the way it deals with modules that have multiple packages in them.
This is not unique to this module, if you search for 
Permission denied at /usr/share/perl/5.22/base.pm line 97
You will see  others doing it too

EG
 I've noticed that Object::InsideOut v3.98 fails when the current directory is not readable by the user running the Perl binary.  Most Perl modules don't have this requirement, and so OIO creates a very cryptic error message when you run it in such circumstances.

However Authen::CAS::Client is a module we use, that is loaded by one of the modules that process_message_queue.pl uses. 
By adding cd /tmp &&  to the cron we make sure we avoid the not readable part.

Of course running as root gets round this too.
This is not a fix, like I said, but a simple workaround so that notices get sent in Koha on perl 5.22. Until this module is replaced (which it is going to be) or another solution involving changing how koha-for-each works under cron (it works fine on the command line) is done.

This does only occur on perl 5.22 and up, so something changed in that.
Comment 24 Julian Maurice 2017-05-30 08:24:36 UTC
I've made a fresh install on Debian stretch (Perl 5.24) but I can't reproduce the problem.

root@koha-stretch:~# sudo -u koha sh -c 'touch test'
touch: cannot touch 'test': Permission denied
root@koha-stretch:~# sudo -u koha sh -c 'ls'
ls: cannot open directory '.': Permission denied

As you can see I have no read nor write access to this directory, but when I execute

root@koha-stretch:~# sudo -u koha sh -c 'PERL5LIB=/home/koha/koha KOHA_CONF=/home/koha/etc/koha-conf.xml perl /home/koha/koha/misc/cronjobs/process_message_queue.pl'

I have no errors and the message queue is processed.

Can you tell me what I'm doing wrong ?
Comment 25 Chris Cormack 2017-05-30 08:31:09 UTC
(In reply to Julian Maurice from comment #24)

> 
> I have no errors and the message queue is processed.
> 
> Can you tell me what I'm doing wrong ?

You are not running it as cron, you are running it from the commandline. The error only manifests under cron.
Comment 26 Julian Maurice 2017-05-30 09:13:43 UTC
# cat /etc/cron.d/koha
* * * * * root sudo -u koha sh -c 'PERL5LIB=/home/koha/koha KOHA_CONF=/home/koha/etc/koha-conf.xml perl /home/koha/koha/misc/cronjobs/process_message_queue.pl > /tmp/koha.log 2>&1'
* * * * * root sudo -u koha sh -c 'pwd > /tmp/pwd.log 2>&1'

/tmp/pwd.log contains "/root"
/tmp/koha.log is empty, and the message queue is processed
Comment 27 Chris Cormack 2017-05-30 09:19:12 UTC
I feel like you are not reading the history of the bug, it is when using the cron job from the packages.

You are not running that.

Step 3 of the test plan says

3/ Build and install a new package (or manually update the
/etc/cron.d/koha-common file)

Not manually run some other cron job.

I am going to bed now, and looks like it has missed the release, so you will have time to read up yourself.
Comment 28 Julian Maurice 2017-05-30 09:31:51 UTC
I've read the history of the bug, many times, and I understand that the problem is with the package install, which I'm trying to mimic to understand what's happening.

If my command succeeds, why the one with koha-foreach should fail ?

Anyway I'll try with a package install later when I have more time
Comment 29 Julian Maurice 2017-05-30 10:23:52 UTC
Just tried a package install following instructions on https://wiki.koha-community.org/wiki/Debian.
Same result. The cron ran process_message_queue.pl and the message queue was processed. No errors.

Should I have specific data in message_queue to trigger the error ?
Comment 30 Katrin Fischer 2017-05-31 05:22:45 UTC
Did you run it manually in your package installation or using the standard cron.d entry?
*/15 * * * * root koha-foreach --enabled --email /usr/share/koha/bin/cronjobs/process_message_queue.pl

Looking at the history of the bug a lot of people have run into it by now - I think there must be a difference in how you test.
Comment 31 Julian Maurice 2017-05-31 07:15:47 UTC
I let the cron do it, as Chris suggested in comment 25.
The only thing I modified in the cron entry is '*/15' to get it launched every minute
Comment 32 Marcel de Rooy 2017-05-31 11:10:19 UTC
Is there a difference between the standard koha-foreach with a sudo koha user and the sudo you added in your cron job ?
Are the permissions of your koha user identical ?
Comment 33 Julian Maurice 2017-05-31 11:33:48 UTC
(In reply to Marcel de Rooy from comment #32)
> Is there a difference between the standard koha-foreach with a sudo koha
> user and the sudo you added in your cron job ?
> Are the permissions of your koha user identical ?
I'm not sure what you are talking about. I used the /etc/cron.d/koha-common shipped with the debian package, and I didn't change the permissions of the 'koha1-koha' user created by koha-create.

I didn't modify the sudoers file either (http://paste.koha-community.org/435)
Comment 34 Julian Maurice 2017-05-31 11:38:04 UTC
More info:

$ pmvers Authen::CAS::Client
0.07

$ pmvers Authen::CAS::Client::Response
0.03

$ perl -v

This is perl 5, version 24, subversion 1 (v5.24.1) built for x86_64-linux-gnu-thread-multi

$ lsb_release -ds
Debian GNU/Linux 9.0 (stretch)
Comment 35 Marcel de Rooy 2017-05-31 11:46:21 UTC
And what about the difference between perl 5.22 and 5.24 ?
Comment 36 Julian Maurice 2017-05-31 12:08:34 UTC
In comment 23, Chris says:
> With perl 5.22 with latest ubuntu, debian testing and above, etc have.
> Something has changed in the way it deals with modules that have multiple
> packages in them.
and

> This does only occur on perl 5.22 and up, so something changed in that.

Since Debian testing have Perl 5.24 I assumed I could reproduce the problem with this version.

I will test with ubuntu 16.10 which has Perl 5.22. Let's hope I reproduce the problem this time :)
Comment 37 Julian Maurice 2017-05-31 12:36:11 UTC
Same with Ubuntu 16.10/Perl 5.22/koha-common 16.11.08

I'm running out of ideas... Does any other QAer want to try ? :)
Comment 38 Julian Maurice 2017-05-31 13:07:12 UTC
I can reproduce a similar error with this command:

root@koha-yakkety:~# sudo -u koha1-koha PERL5LIB=. perl -MAuthen::CAS::Client -E 'say "Hi"'
Can't locate Authen/CAS/Client.pm:   ./Authen/CAS/Client.pm: Permission denied.
BEGIN failed--compilation aborted.

root@koha-yakkety:~# cd /tmp/
root@koha-yakkety:/tmp# sudo -u koha1-koha PERL5LIB=. perl -MAuthen::CAS::Client -E 'say "Hi"'
Hi

So, people that can reproduce the problem, could you please paste the output of the following command (at least the @INC part):

# As root
sudo -u <KOHA_USER> perl -V
Comment 39 Tomás Cohen Arazi 2017-05-31 14:13:19 UTC
Ok, I managed to reproduce the conditions that trigger the bug.

GOOD NEWS: It is fixed in master, 16.11 and 16.05 already, as a side effect from bug 16733.

The problem was in koha-foreach code, that is not using koha-shell but crafting its own env instead. To reproduce the problem, as a root user run:

root$ echo $PERL5LIB
<empty>
root$ . /usr/share/koha/bin/koha-functions.sh
root$ adjust_paths_dev_install kohadev
root$ echo $PERL5LIB
/home/vagrant/kohaclone

So, if adjust_paths_dev_install is not ran, and the command is run by root, PERL5LIB is empty, and perl uses the fallback for include (i.e. '.').

This are the conditions in which cron.d/koha-common runs process_message_queue.pl. This happened in koha-foreach code, before bug 16733. This is the relevant code in debian/scripts/koha-foreach:

for name in $(koha-list $listopts)
do
    # Optionally use alternative paths for a dev install
    adjust_paths_dev_install $name

    cmd=`echo "$@" | sed -e s/__instancename__/${name}/g`
    (
    exec 3>&1
    sudo -u "$name-koha" \
    env PERL5LIB=$PERL5LIB \
        KOHA_CONF="/etc/koha/sites/$name/koha-conf.xml" \
        ${cmd} 2>&1 >&3 | sed -e "s/^/$name: /" >&2 3>&-
    exec 3>&-
    ) | sed -e "s/^/$name: /"

done

So before the 16733 backport, PERL5LIB was incorrectly set. This script should definitely run koha-shell -c "${cmd}" $name instead of crafting all the environment variables.
Comment 40 Marco Moreno 2017-08-08 21:56:32 UTC
This issue is NOT fixed in 17.05.  I'm seeing this now after having upgraded from 16.11.  However, I've discovered some things that might be helpful.

As Chris mentioned, this appears to be a bug with base.pm (IMO).  Perhaps it's behavior has changed due to this security resolution that forces Perl to die if a required module is cannot be read when traversing @INC:
https://rt.perl.org/Public/Bug/Display.html?id=113422

According to this post, base.pm may attempt to require a module that does not exist and traverse the entire @INC array:
https://rt.cpan.org/Public/Bug/Display.html?id=106552#txn-1589169
Evidently, it dies as soon as it encounters a directory it cannot read.

Avoiding '.' in PERL5LIB doesn't really help since Perl already includes '.' in @INC (also a security concern???).  Working around this means either ensuring that '.' points to a directory that can be read (e.g. "cd /tmp") or modifying the @INC array to remove '.' from it.

The latter seems better so I modified the BEGIN block in /usr/share/koha/bin/kohalib.pl like this:

-------------------------------------
use strict;
#use warnings; FIXME - Bug 2505

my $module_dir;
BEGIN {
    $module_dir = '/usr/share/koha/lib';
    die if $module_dir =~ /^[_]{2}PERL_MODULE_DIR[_]{2}$/;
    @INC = grep {$_ ne '.'} @INC;
}

use lib $module_dir;

1;
-------------------------------------

This also allows the change to be in one place and avoid having to insert a 'cd /tmp' for each cron job.

I'm running Perl 5.22 - I wonder if it's fixed in 5.24.
Comment 41 Tomás Cohen Arazi 2017-08-11 18:22:46 UTC
(In reply to Marco Moreno from comment #40)
> This issue is NOT fixed in 17.05.  I'm seeing this now after having upgraded
> from 16.11.  However, I've discovered some things that might be helpful.

Can you mention your steps to reproduce?
Comment 42 Marco Moreno 2017-08-11 18:47:25 UTC
ubuntu@koha:~$ sudo -i

root@koha:~# sudo -u library-koha perl -MAuthen::CAS::Client -e 'print "Ok\n"'
Can't locate Authen/CAS/Client/Response/Failure.pm:   ./Authen/CAS/Client/Response/Failure.pm: Permission denied at /usr/share/perl/5.22/base.pm line 97.
	...propagated at /usr/share/perl/5.22/base.pm line 106.
BEGIN failed--compilation aborted at /usr/share/perl5/Authen/CAS/Client/Response.pm line 68.
Compilation failed in require at /usr/share/perl5/Authen/CAS/Client.pm line 8.
BEGIN failed--compilation aborted at /usr/share/perl5/Authen/CAS/Client.pm line 8.
Compilation failed in require.
BEGIN failed--compilation aborted.

root@koha:~# cd /tmp

root@koha:/tmp# sudo -u library-koha perl -MAuthen::CAS::Client -e 'print "Ok\n"'
Ok


root@koha:~# perl -Mbase -e 'print "$base::VERSION\n"'
2.22

root@koha:~# perl -V
Summary of my perl5 (revision 5 version 22 subversion 1) configuration:
   
  Platform:
    osname=linux, osvers=3.16.0, archname=x86_64-linux-gnu-thread-multi
    uname='linux localhost 3.16.0 #1 smp debian 3.16.0 x86_64 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dcc=x86_64-linux-gnu-gcc -Dcpp=x86_64-linux-gnu-cpp -Dld=x86_64-linux-gnu-gcc -Dccflags=-DDEBIAN -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Dldflags= -Wl,-Bsymbolic-functions -Wl,-z,relro -Dlddlflags=-shared -Wl,-Bsymbolic-functions -Wl,-z,relro -Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.22 -Darchlib=/usr/lib/x86_64-linux-gnu/perl/5.22 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/x86_64-linux-gnu/perl5/5.22 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.22.1 -Dsitearch=/usr/local/lib/x86_64-linux-gnu/perl/5.22.1 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Duse64bitint -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -Ui_libutil -Uversiononly -DDEBUGGING=-g -Doptimize=-O2 -dEs -Duseshrplib -Dlibperl=libperl.so.5.22.1'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='x86_64-linux-gnu-gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -g',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include'
    ccversion='', gccversion='5.3.1 20160311', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='x86_64-linux-gnu-gcc', ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/5/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=libc-2.21.so, so=so, useshrplib=true, libperl=libperl.so.5.22
    gnulibc_version='2.21'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib -fstack-protector-strong'


Characteristics of this binary (from libperl): 
  Compile-time options: HAS_TIMES MULTIPLICITY PERLIO_LAYERS
                        PERL_DONT_CREATE_GVSV
                        PERL_HASH_FUNC_ONE_AT_A_TIME_HARD
                        PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP
                        PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV
                        USE_64_BIT_ALL USE_64_BIT_INT USE_ITHREADS
                        USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE
                        USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME
                        USE_PERLIO USE_PERL_ATOF USE_REENTRANT_API
.
.
.
  Built under linux
  Compiled at Mar 13 2016 11:54:18
  @INC:
    /etc/perl
    /usr/local/lib/x86_64-linux-gnu/perl/5.22.1
    /usr/local/share/perl/5.22.1
    /usr/lib/x86_64-linux-gnu/perl5/5.22
    /usr/share/perl5
    /usr/lib/x86_64-linux-gnu/perl/5.22
    /usr/share/perl/5.22
    /usr/local/lib/site_perl
    /usr/lib/x86_64-linux-gnu/perl-base
    .
Comment 43 Marco Moreno 2017-08-11 19:41:54 UTC
Here is another way that demonstrates the fix of modifying the @INC array similarly to how I described above:

root@koha:~# cd /root

root@koha:~# sudo -u library-koha perl -e 'use Authen::CAS::Client; print "Ok\n"'
Can't locate Authen/CAS/Client/Response/Failure.pm:   ./Authen/CAS/Client/Response/Failure.pm: Permission denied at /usr/share/perl/5.22/base.pm line 97.
	...propagated at /usr/share/perl/5.22/base.pm line 106.
BEGIN failed--compilation aborted at /usr/share/perl5/Authen/CAS/Client/Response.pm line 68.
Compilation failed in require at /usr/share/perl5/Authen/CAS/Client.pm line 8.
BEGIN failed--compilation aborted at /usr/share/perl5/Authen/CAS/Client.pm line 8.
Compilation failed in require at -e line 1.
BEGIN failed--compilation aborted at -e line 1.


root@koha:~# sudo -u library-koha perl -e 'BEGIN {pop @INC} use Authen::CAS::Client; print "Ok\n"'
Ok
Comment 44 Magnus Enger 2017-11-16 08:17:46 UTC
I ran into this problem after upgrading from

Koha 3.22, Ubuntu 14.04, Perl ?

to 

Koha 17.05.05, Ubuntu 16.04.3 LTS, Perl v5.22.1

gather_print_notices.pl run from cron (sudo crontab -e) would fail with this error:

Can't locate Koha/Exceptions/Exception.pm: ./Koha/Exceptions/Exception.pm: Permission denied at /usr/share/perl/5.22/base.pm line 97.
...propagated at /usr/share/perl/5.22/base.pm line 106.
BEGIN failed--compilation aborted at (eval 14) line 3.
BEGIN failed--compilation aborted at /usr/share/koha/lib/Koha/Exceptions.pm line 56.
Compilation failed in require at /usr/share/koha/lib/Koha/DateUtils.pm line 22.
BEGIN failed--compilation aborted at /usr/share/koha/lib/Koha/DateUtils.pm line 22.
Compilation failed in require at /usr/share/koha/lib/C4/Log.pm line 30.
BEGIN failed--compilation aborted at /usr/share/koha/lib/C4/Log.pm line 30.
Compilation failed in require at /usr/share/koha/lib/Koha/Config/SysPref.pm line 26.
BEGIN failed--compilation aborted at /usr/share/koha/lib/Koha/Config/SysPref.pm line 26.
Compilation failed in require at /usr/share/koha/lib/C4/Context.pm line 105.
BEGIN failed--compilation aborted at /usr/share/koha/lib/C4/Context.pm line 105.
Compilation failed in require at /usr/share/koha/bin/cronjobs/gather_print_notices.pl line 13.
BEGIN failed--compilation aborted at /usr/share/koha/bin/cronjobs/gather_print_notices.pl line 13.

Running the command in the crontab on the command line would succeed, no errors. 

Adding the "cd /tmp &&" trick in front of the commands in the crontab makes the errors go away.
Comment 45 Larry Baerveldt 2017-12-09 03:37:24 UTC
We're getting the exact same error when trying to do a full rebuild from root cron, using /usr/sbin/koha-rebuild-zebra. 

Koha 17.05.05, Ubuntu 16.04.3 LTS, Perl v5.22.1
Cronjob: /usr/sbin/koha-rebuild-zebra -f -v SITE

Error:
Rebuilding authority indexes.
Can't locate Koha/Exceptions/Exception.pm:   ./Koha/Exceptions/Exception.pm: Permission denied at /usr/share/perl/5.22/base.pm line 97.
	...propagated at /usr/share/perl/5.22/base.pm line 106.
BEGIN failed--compilation aborted at (eval 15) line 3.
BEGIN failed--compilation aborted at /usr/share/koha/lib/Koha/Exceptions.pm line 56.
Compilation failed in require at /usr/share/koha/lib/Koha/DateUtils.pm line 22.
BEGIN failed--compilation aborted at /usr/share/koha/lib/Koha/DateUtils.pm line 22.
Compilation failed in require at /usr/share/koha/lib/C4/Log.pm line 30.
BEGIN failed--compilation aborted at /usr/share/koha/lib/C4/Log.pm line 30.
Compilation failed in require at /usr/share/koha/lib/Koha/Config/SysPref.pm line 26.
BEGIN failed--compilation aborted at /usr/share/koha/lib/Koha/Config/SysPref.pm line 26.
Compilation failed in require at /usr/share/koha/lib/C4/Context.pm line 105.
BEGIN failed--compilation aborted at /usr/share/koha/lib/C4/Context.pm line 105.
Compilation failed in require at /usr/share/koha/bin/migration_tools/rebuild_zebra.pl line 20.
BEGIN failed--compilation aborted at /usr/share/koha/bin/migration_tools/rebuild_zebra.pl line 20.
Something went wrong rebuilding authority indexes for library
Wed Dec  6 22:25:41 PST 2017
 
Rebuilding biblios indexes.
Can't locate Koha/Exceptions/Exception.pm:   ./Koha/Exceptions/Exception.pm: Permission denied at /usr/share/perl/5.22/base.pm line 97.
	...propagated at /usr/share/perl/5.22/base.pm line 106.
BEGIN failed--compilation aborted at (eval 15) line 3.
BEGIN failed--compilation aborted at /usr/share/koha/lib/Koha/Exceptions.pm line 56.
Compilation failed in require at /usr/share/koha/lib/Koha/DateUtils.pm line 22.
BEGIN failed--compilation aborted at /usr/share/koha/lib/Koha/DateUtils.pm line 22.
Compilation failed in require at /usr/share/koha/lib/C4/Log.pm line 30.
BEGIN failed--compilation aborted at /usr/share/koha/lib/C4/Log.pm line 30.
Compilation failed in require at /usr/share/koha/lib/Koha/Config/SysPref.pm line 26.
BEGIN failed--compilation aborted at /usr/share/koha/lib/Koha/Config/SysPref.pm line 26.
Compilation failed in require at /usr/share/koha/lib/C4/Context.pm line 105.
BEGIN failed--compilation aborted at /usr/share/koha/lib/C4/Context.pm line 105.
Compilation failed in require at /usr/share/koha/bin/migration_tools/rebuild_zebra.pl line 20.
BEGIN failed--compilation aborted at /usr/share/koha/bin/migration_tools/rebuild_zebra.pl line 20.
Something went wrong rebuilding biblio indexes for library

The same work-around works:
cd /tmp && /usr/sbin/koha-rebuild-zebra -f -v SITE
Comment 46 Katrin Fischer 2017-12-10 21:14:20 UTC
It looks like this one is not dead yet - Tomas, could you take another look as you hunted down the problem last time?
Comment 47 Marco Moreno 2017-12-15 06:14:54 UTC
Understanding the problem is necessary to determine the best solution.

The problem is this:

- All cron jobs start in $HOME and those in /etc/cron* run as root which is /root.
- Problems occur when using "sudo -u another_user" to demote from root to "another_user" which does not have read access to /root.
- The @INC array is traversed when a module is to be loaded, but a recent update to Perl 5.22 (base.pm?) now results in a fatal error when it searches a directory that cannot be read.
- The @INC array includes the current directory ('.') which is '/root' and unreadable by 'another_user', resulting in this error.
- This error will also occur if run from the command line in a directory which is not readable my 'another_user'.

Normally, crons are set up in the crontab file for the user under which it should run. 
 Since it is rather unusual for a cron to sudo to another user, this is likely why this regression in Perl has gone largely unnoticed.

Possible solutions:

1. Fix the regression in Perl 5.22.  This is ideal and this may have already happened in more recent versions of Perl.  However, many of us remain stuck with the version we have and we need a solution that will with with a broken Perl.  Not a good short-term option.

2. Avoid using "sudo -u".  While possibly ideal, it's convenient and simple for koha-foreach to be run as a system cron and switch to the user for each library. Therefore, this is not a good option.

3. Modify the @INC array to remove the '.' directory so that /root will never be searched.  This could be done in a common location (e.g. /usr/share/koha/bin/kohalib.pl) as I suggested in Comment 40.  This is risky because it assumes that no downstream module will ever depend upon '.' having been removed from @INC.  While this solution has worked well for me, it probably is not ideal either.

4. Change the current directory to one that is readable by adding 'cd /tmp' to each cron job that performs a "sudo -u".  This has become an established workaround, but requires modifying many cron entries.  Also not an ideal solution.

5. Change the current directory to one that is readable by modifying all code that performs a "sudo -u".  This may be as simple as adding "cd /tmp" at the beginning of /usr/sbin/koha-foreach and /usr/sbin/koha-rebuild-zebra.  Possibly the best solution for now.

Option #5 seems like the best one for now.  It is a clear, one-line solution that doesn't require modifying any crons.

Thoughts?
Comment 48 Marcel de Rooy 2017-12-15 06:49:23 UTC
(In reply to Marco Moreno from comment #47)
> Understanding the problem is necessary to determine the best solution.
[...]
> 5. Change the current directory to one that is readable by modifying all
> code that performs a "sudo -u".  This may be as simple as adding "cd /tmp"
> at the beginning of /usr/sbin/koha-foreach and /usr/sbin/koha-rebuild-zebra.
> Possibly the best solution for now.
> 
> Option #5 seems like the best one for now.  It is a clear, one-line solution
> that doesn't require modifying any crons.

I would agree. Put cd /tmp very close to the sudo -u. Adding it in cron is more confusing.
Clear argumentation imo!
Comment 49 Chris Nighswonger 2018-02-19 20:24:09 UTC
I can confirm this is still an issue on:

Koha 17.11.01.000, Perl 5.22.1, Ubuntu 16.04.3 LTS

Chris C's workaround fixes it for me.
Comment 50 Jonathan Druart 2018-02-21 13:59:53 UTC
Can anyone take care of this one?
Comment 51 Tomás Cohen Arazi 2018-03-02 14:35:39 UTC
(In reply to Jonathan Druart from comment #50)
> Can anyone take care of this one?

I'm taking care of it. I'm working on the same approach as bug 19546. Adding a --chdir option switch to koha-foreach.
Comment 52 Tomás Cohen Arazi 2018-03-02 14:36:57 UTC
(In reply to Marcel de Rooy from comment #48)
> (In reply to Marco Moreno from comment #47)
> > Understanding the problem is necessary to determine the best solution.
> [...]
> > 5. Change the current directory to one that is readable by modifying all
> > code that performs a "sudo -u".  This may be as simple as adding "cd /tmp"
> > at the beginning of /usr/sbin/koha-foreach and /usr/sbin/koha-rebuild-zebra.
> > Possibly the best solution for now.
> > 
> > Option #5 seems like the best one for now.  It is a clear, one-line solution
> > that doesn't require modifying any crons.
> 
> I would agree. Put cd /tmp very close to the sudo -u. Adding it in cron is
> more confusing.
> Clear argumentation imo!

If there's a /tmp/Koha directory, without read permissions for the instance's user it will still fail. I think we should jump into the instance's home directory and I'm providing a patch to do so.
Comment 53 Tomás Cohen Arazi 2018-03-02 15:20:24 UTC
Created attachment 72368 [details] [review]
Bug 17717: Add a --chdir option switch for koha-foreach

Until Perl 5.26, the current directory is added to @INC when running a
Perl script [1]. Having the current directory in @INC means it can be
tried to be traversed when performing a lib lookup. Since version 5.18,
Perl dies when it finds an unreadable directory (permissions) in @INC
that needs to be traversed. This behaviour won't change because Perl
devs consider it an enhancement to security. [2]

Because of this, we need to make sure our scripts are ran **from** a
directory in which they have read permissions.

Ths patch adds a --chdir option switch to the **koha-foreach** wrapper
script, that makes the inner shells/scripts to be ran within the Koha
instance's user home directory.

The change is trivial and should be QAed easily. I tested this on a prod
server:

- Create a /tmp/test.pl file containing:

use Modern::Perl;

use Cwd;
my $dir = getcwd;

warn $dir;

1;

A) then create a cronjob entry to run it using koha-foreach:
(in /etc/cron.d/test):
1/* * * * * root koha-foreach perl /tmp/test.pl
- Once I noticed the cronjob ran, I used mutt to read the emails in the
root user.
=> FAIL:
...
Subject: Cron <root@koha> koha-foreach --enabled perl /tmp/test.pl

"/root"
"/root"
"/root"
"/root"
"/root"
...

B) I then used the patched koha-foreach with different results:
=> SUCCESS:
...
Subject: Cron <root@koha> /root/koha-foreach --chdir --enabled perl /tmp/test.pl

"/var/lib/koha/acaderc"
"/var/lib/koha/agro"
"/var/lib/koha/anc"
"/var/lib/koha/arico"
"/var/lib/koha/artes"
...

So this patch's approach works. But...

C) master's koha-foreach seems to work just the same... I think it is
because of my previous attempt to fix this by using sudo in koha-shell.
So I think environmental conditions affect the behaviour (which shell is
configured for cron, sudo configuration, etc).

====

In conclusion, I think we should go ahead with this patch as it will solve
peoples issues, and it is a right solution (option #5 on the list) to
this Perl behaviour change. It doesn't cover other commands, but
followup patches could do.

I avoided /tmp as it is writable by any user... so it is an easy path
for both exploiting by replacing some lib, and also because the
existence of an unreadable dir that the interpreter could try to
traverse (unreadable /tmp/Authen or /tmp/Koha will trigger the same
error, and I assume people know what they are putting on the instance's
dir, at least it will be easier to track).

A followup patch takes care of making the cronjobs use --chdir when
calling koha-foreach

[1] https://lists.debian.org/debian-devel-announce/2016/08/msg00013.html
[2] https://rt.perl.org/Public/Bug/Display.html?id=123795
Comment 54 Tomás Cohen Arazi 2018-03-02 15:20:33 UTC
Created attachment 72369 [details] [review]
Bug 17717: Make cronjobs using koha-foreach use --chdir

In order to patch production sites we need to adjust the shipped
cronjobs so they are called with the --chdir option switch.
Comment 55 Kyle M Hall 2018-03-02 15:50:15 UTC
Created attachment 72372 [details] [review]
Bug 17717: Add a --chdir option switch for koha-foreach

Until Perl 5.26, the current directory is added to @INC when running a
Perl script [1]. Having the current directory in @INC means it can be
tried to be traversed when performing a lib lookup. Since version 5.18,
Perl dies when it finds an unreadable directory (permissions) in @INC
that needs to be traversed. This behaviour won't change because Perl
devs consider it an enhancement to security. [2]

Because of this, we need to make sure our scripts are ran **from** a
directory in which they have read permissions.

Ths patch adds a --chdir option switch to the **koha-foreach** wrapper
script, that makes the inner shells/scripts to be ran within the Koha
instance's user home directory.

The change is trivial and should be QAed easily. I tested this on a prod
server:

- Create a /tmp/test.pl file containing:

use Modern::Perl;

use Cwd;
my $dir = getcwd;

warn $dir;

1;

A) then create a cronjob entry to run it using koha-foreach:
(in /etc/cron.d/test):
1/* * * * * root koha-foreach perl /tmp/test.pl
- Once I noticed the cronjob ran, I used mutt to read the emails in the
root user.
=> FAIL:
...
Subject: Cron <root@koha> koha-foreach --enabled perl /tmp/test.pl

"/root"
"/root"
"/root"
"/root"
"/root"
...

B) I then used the patched koha-foreach with different results:
=> SUCCESS:
...
Subject: Cron <root@koha> /root/koha-foreach --chdir --enabled perl /tmp/test.pl

"/var/lib/koha/acaderc"
"/var/lib/koha/agro"
"/var/lib/koha/anc"
"/var/lib/koha/arico"
"/var/lib/koha/artes"
...

So this patch's approach works. But...

C) master's koha-foreach seems to work just the same... I think it is
because of my previous attempt to fix this by using sudo in koha-shell.
So I think environmental conditions affect the behaviour (which shell is
configured for cron, sudo configuration, etc).

====

In conclusion, I think we should go ahead with this patch as it will solve
peoples issues, and it is a right solution (option #5 on the list) to
this Perl behaviour change. It doesn't cover other commands, but
followup patches could do.

I avoided /tmp as it is writable by any user... so it is an easy path
for both exploiting by replacing some lib, and also because the
existence of an unreadable dir that the interpreter could try to
traverse (unreadable /tmp/Authen or /tmp/Koha will trigger the same
error, and I assume people know what they are putting on the instance's
dir, at least it will be easier to track).

A followup patch takes care of making the cronjobs use --chdir when
calling koha-foreach

[1] https://lists.debian.org/debian-devel-announce/2016/08/msg00013.html
[2] https://rt.perl.org/Public/Bug/Display.html?id=123795

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 56 Kyle M Hall 2018-03-02 15:50:24 UTC
Created attachment 72373 [details] [review]
Bug 17717: Make cronjobs using koha-foreach use --chdir

In order to patch production sites we need to adjust the shipped
cronjobs so they are called with the --chdir option switch.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 57 Marco Moreno 2018-03-02 16:01:18 UTC
Hmmm...I'm now reconsidering this and wonder if option #3 is really the best solution by removing '.' from @INC.

You made a good point about /tmp being a concern.  This, plus the fact that they have removed '.' from @INC in recent versions of Perl, has convinced me that having '.' in @INC is generally a very bad idea and a major security concern.

Therefore, I want to propose revisiting comment #40 which removes '.' from @INC in a common library early in the bootstrapping process.  This effectively undoes the "feature" added in Perl 5.18 and removed in Perl 5.26.  Additionally, this prevents exploits that attempt to insert '.' via PERL5LIB.

It is a single line of code, does nothing if '.' doesn't exist in @INC, and doesn't require modifying crons.
Comment 58 Tomás Cohen Arazi 2018-03-02 16:13:18 UTC
(In reply to Marco Moreno from comment #57)
> Hmmm...I'm now reconsidering this and wonder if option #3 is really the best
> solution by removing '.' from @INC.
> 
> You made a good point about /tmp being a concern.  This, plus the fact that
> they have removed '.' from @INC in recent versions of Perl, has convinced me
> that having '.' in @INC is generally a very bad idea and a major security
> concern.
> 
> Therefore, I want to propose revisiting comment #40 which removes '.' from
> @INC in a common library early in the bootstrapping process.  This
> effectively undoes the "feature" added in Perl 5.18 and removed in Perl
> 5.26.  Additionally, this prevents exploits that attempt to insert '.' via
> PERL5LIB.

I considered that option. But we would need to do it on every script, and we would also be changing the current behaviour. Someone might be using their own lib (or version of some lib) on purpose and we'd be breaking them. So, for the general user base, we should stick with the cleaner solution IMHO.
Comment 59 Marco Moreno 2018-03-02 16:27:20 UTC
Couldn't it be done in once in a common library that is used everywhere, like /usr/share/koha/bin/kohalib.pl?

Sorry, I really don't know much about Koha's architecture and I trust you've studied this more than I have.
Comment 60 Tomás Cohen Arazi 2018-03-02 18:31:50 UTC
(In reply to Marco Moreno from comment #59)
> Couldn't it be done in once in a common library that is used everywhere,
> like /usr/share/koha/bin/kohalib.pl?
> 
> Sorry, I really don't know much about Koha's architecture and I trust you've
> studied this more than I have.

I understand the kohalib.pl approach, but I think it is too hacky and I don't think we should use it at all. So a solution there I wouldn't agree. Also, not all CLI scripts written in perl use it...
Comment 61 Marco Moreno 2018-03-02 18:48:02 UTC
Fair enough - trust your judgment.
Comment 62 Marcel de Rooy 2018-03-05 10:27:48 UTC
Looking here. Something wrong still
Comment 63 Marcel de Rooy 2018-03-05 10:36:13 UTC
Created attachment 72411 [details] [review]
Bug 17717: Add a --chdir option switch for koha-foreach

Until Perl 5.26, the current directory is added to @INC when running a
Perl script [1]. Having the current directory in @INC means it can be
tried to be traversed when performing a lib lookup. Since version 5.18,
Perl dies when it finds an unreadable directory (permissions) in @INC
that needs to be traversed. This behaviour won't change because Perl
devs consider it an enhancement to security. [2]

Because of this, we need to make sure our scripts are ran **from** a
directory in which they have read permissions.

Ths patch adds a --chdir option switch to the **koha-foreach** wrapper
script, that makes the inner shells/scripts to be ran within the Koha
instance's user home directory.

The change is trivial and should be QAed easily. I tested this on a prod
server:

- Create a /tmp/test.pl file containing:

use Modern::Perl;

use Cwd;
my $dir = getcwd;

warn $dir;

1;

A) then create a cronjob entry to run it using koha-foreach:
(in /etc/cron.d/test):
1/* * * * * root koha-foreach perl /tmp/test.pl
- Once I noticed the cronjob ran, I used mutt to read the emails in the
root user.
=> FAIL:
...
Subject: Cron <root@koha> koha-foreach --enabled perl /tmp/test.pl

"/root"
"/root"
"/root"
"/root"
"/root"
...

B) I then used the patched koha-foreach with different results:
=> SUCCESS:
...
Subject: Cron <root@koha> /root/koha-foreach --chdir --enabled perl /tmp/test.pl

"/var/lib/koha/acaderc"
"/var/lib/koha/agro"
"/var/lib/koha/anc"
"/var/lib/koha/arico"
"/var/lib/koha/artes"
...

So this patch's approach works. But...

C) master's koha-foreach seems to work just the same... I think it is
because of my previous attempt to fix this by using sudo in koha-shell.
So I think environmental conditions affect the behaviour (which shell is
configured for cron, sudo configuration, etc).

====

In conclusion, I think we should go ahead with this patch as it will solve
peoples issues, and it is a right solution (option #5 on the list) to
this Perl behaviour change. It doesn't cover other commands, but
followup patches could do.

I avoided /tmp as it is writable by any user... so it is an easy path
for both exploiting by replacing some lib, and also because the
existence of an unreadable dir that the interpreter could try to
traverse (unreadable /tmp/Authen or /tmp/Koha will trigger the same
error, and I assume people know what they are putting on the instance's
dir, at least it will be easier to track).

A followup patch takes care of making the cronjobs use --chdir when
calling koha-foreach

[1] https://lists.debian.org/debian-devel-announce/2016/08/msg00013.html
[2] https://rt.perl.org/Public/Bug/Display.html?id=123795

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 64 Marcel de Rooy 2018-03-05 10:36:20 UTC
Created attachment 72412 [details] [review]
Bug 17717: Make cronjobs using koha-foreach use --chdir

In order to patch production sites we need to adjust the shipped
cronjobs so they are called with the --chdir option switch.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 65 Marcel de Rooy 2018-03-05 10:36:26 UTC
Created attachment 72413 [details] [review]
Bug 17717: (QA follow-up) Fix typo chdir

This test does obviously not achieve the desired result:
    [ "chdir" != "no" ]
Trivial fix.
Adding the same quotes around starting_dir (just as for Bug 19546).

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 66 Nick Clemens 2018-03-15 09:22:53 UTC
Pushed to master for 18.05, thanks to all for your hard work.
Comment 67 Nick Clemens 2018-04-06 17:52:14 UTC
Awesome work all, backported to stable for 17.11.05
Comment 68 Fridolin Somers 2018-04-10 07:17:43 UTC
Pushed to 17.05.x for v17.05.11
Comment 69 Katrin Fischer 2019-02-10 13:13:07 UTC
*** Bug 20786 has been marked as a duplicate of this bug. ***
Comment 70 Nitesh Kumar Verma 2021-04-27 08:59:55 UTC
Facing the same problem in my KOHA version 16.11. While trying to apply the said patch, I am not able to find the file 'koha-foreach.xml'.

Guide me to find the file koha-foreach.xml so that may apply the patch.

Regards,
Dr. Nitesh Kumar Verma
Comment 71 Katrin Fischer 2021-04-28 22:32:36 UTC
Hi Nitesh,
this bug has been fixed in 17.05 and later. 16.11 is no longer maintained and won't receive any updates. Please consider updating. 
If that's not an option asking on the mailing liston how to make this change locally would be better.