Bug 16733 - More flexible paths in debian scripts (for dev installs)
Summary: More flexible paths in debian scripts (for dev installs)
Status: Pushed to Stable
Alias: None
Product: Koha
Classification: Unclassified
Component: Packaging (show other bugs)
Version: master
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Marcel de Rooy
QA Contact: Testopia
URL:
Whiteboard:
Keywords:
: 13971 (view as bug list)
Depends on: 16644
Blocks: 16749 17049
  Show dependency treegraph
 
Reported: 2016-06-14 14:19 UTC by Marcel de Rooy
Modified: 2017-05-24 03:06 UTC (History)
8 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Small patch
Bot Control: ---
When did the bot last check this:
Who signed the patch off:
Text to go in the release notes:


Attachments
Bug 16733: Add adjust_paths_dev_install to koha-functions.sh (2.09 KB, patch)
2016-06-16 14:24 UTC, Marcel de Rooy
Details | Diff | Splinter Review
Bug 16733: Adjust koha-indexer (1.80 KB, patch)
2016-06-16 14:24 UTC, Marcel de Rooy
Details | Diff | Splinter Review
Bug 16733: Adjust koha-rebuild-zebra (1.92 KB, patch)
2016-06-16 14:25 UTC, Marcel de Rooy
Details | Diff | Splinter Review
Bug 16733: Adjust debian scripts for hardcoded PERL5LIBs (10.79 KB, patch)
2016-06-16 14:25 UTC, Marcel de Rooy
Details | Diff | Splinter Review
Bug 16733: Adjust other debian scripts using PERL5LIB (11.74 KB, patch)
2016-06-20 11:19 UTC, Marcel de Rooy
Details | Diff | Splinter Review
Bug 16733: Adjust other debian scripts using PERL5LIB (11.89 KB, patch)
2016-06-20 11:24 UTC, Marcel de Rooy
Details | Diff | Splinter Review
[SIGNED-OFF] Bug 16733: Add adjust_paths_dev_install to koha-functions.sh (2.15 KB, patch)
2016-07-13 16:15 UTC, Owen Leonard
Details | Diff | Splinter Review
[SIGNED-OFF] Bug 16733: Adjust koha-indexer (1.86 KB, patch)
2016-07-13 16:15 UTC, Owen Leonard
Details | Diff | Splinter Review
[SIGNED-OFF] Bug 16733: Adjust koha-rebuild-zebra (1.98 KB, patch)
2016-07-13 16:16 UTC, Owen Leonard
Details | Diff | Splinter Review
[SIGNED-OFF] Bug 16733: Adjust other debian scripts using PERL5LIB (11.94 KB, patch)
2016-07-13 16:16 UTC, Owen Leonard
Details | Diff | Splinter Review
Bug 16733: [Follow-up] Add $home to api path too (890 bytes, patch)
2016-09-12 10:32 UTC, Marcel de Rooy
Details | Diff | Splinter Review
Bug 16733: Add adjust_paths_dev_install to koha-functions.sh (2.17 KB, patch)
2017-01-18 09:48 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 16733: Adjust koha-indexer (1.88 KB, patch)
2017-01-18 09:49 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 16733: Adjust koha-rebuild-zebra (2.00 KB, patch)
2017-01-18 09:49 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 16733: Adjust other debian scripts using PERL5LIB (11.94 KB, patch)
2017-01-18 09:49 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 16733: [Follow-up] Add $home to api path too (918 bytes, patch)
2017-01-18 09:49 UTC, Jonathan Druart
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Marcel de Rooy 2016-06-14 14:19:13 UTC

    
Comment 1 Marcel de Rooy 2016-06-16 14:24:56 UTC Comment hidden (obsolete)
Comment 2 Marcel de Rooy 2016-06-16 14:24:59 UTC Comment hidden (obsolete)
Comment 3 Marcel de Rooy 2016-06-16 14:25:03 UTC Comment hidden (obsolete)
Comment 4 Marcel de Rooy 2016-06-16 14:25:06 UTC Comment hidden (obsolete)
Comment 5 Marcel de Rooy 2016-06-16 14:26:11 UTC
WIP: Still working on the last patch a little.
Comment 6 Marcel de Rooy 2016-06-20 11:19:53 UTC Comment hidden (obsolete)
Comment 7 Marcel de Rooy 2016-06-20 11:24:40 UTC Comment hidden (obsolete)
Comment 8 Marcel de Rooy 2016-06-20 11:44:17 UTC
*** Bug 13971 has been marked as a duplicate of this bug. ***
Comment 9 Owen Leonard 2016-07-12 14:43:37 UTC
(In reply to Marcel de Rooy from comment #1)

> Test plan:
> [1] Add <dev_install>/not/there</dev_install> to your koha-conf.xml.
> [2] Run on the command line:
>         PERL5LIB=test
>         source [path-to-your-instance]/debian/scripts/koha-functions.sh
>         echo $PERL5LIB
>         adjust_paths_dev_install [name-of-your-instance]

This last step results in an error in my gitified package installation. It says "bash: adjust_paths_dev_install: command not found."
Comment 10 Marcel de Rooy 2016-07-13 12:44:02 UTC
(In reply to Owen Leonard from comment #9)
> (In reply to Marcel de Rooy from comment #1)
> 
> > Test plan:
> > [1] Add <dev_install>/not/there</dev_install> to your koha-conf.xml.
> > [2] Run on the command line:
> >         PERL5LIB=test
> >         source [path-to-your-instance]/debian/scripts/koha-functions.sh
> >         echo $PERL5LIB
> >         adjust_paths_dev_install [name-of-your-instance]
> 
> This last step results in an error in my gitified package installation. It
> says "bash: adjust_paths_dev_install: command not found."

Hi Owen,
Nice to see you test this.
The command not found-response sounds to me that you did not source the right koha-functions file? Somehow the function was not loaded.
You should source the koha-functions file that has been adjusted by this patch.
If I follow the test plan, the output is as expected.
Could you clarify?
Comment 11 Owen Leonard 2016-07-13 16:15:46 UTC
Created attachment 53384 [details] [review]
[SIGNED-OFF] Bug 16733: Add adjust_paths_dev_install to koha-functions.sh

This new function checks koha-conf.xml for a given instance and if it
contains a dev_install line, it adjusts PERL5LIB and KOHA_HOME
accordingly. Otherwise it does not touch the values of these
variables as normally read from /etc/default/koha-common.

The function will be used in various debian scripts to allow for more
flexibility with dev installs. And at the same time aiming to make better
use of PERL5LIB and KOHA_HOME.

Test plan:
[1] Add <dev_install>/not/there</dev_install> to your koha-conf.xml.
[2] Run on the command line:
        PERL5LIB=test
        source [path-to-your-instance]/debian/scripts/koha-functions.sh
        echo $PERL5LIB
        adjust_paths_dev_install [name-of-your-instance]
        echo $PERL5LIB
    The last echo should be: /not/there
[3] Remove the <dev_install> line and repeat step 2.
    The last echo should be: test

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Comment 12 Owen Leonard 2016-07-13 16:15:53 UTC
Created attachment 53385 [details] [review]
[SIGNED-OFF] Bug 16733: Adjust koha-indexer

[1] Add a call to the new adjust_paths_dev_install
[2] Differentiate location of rebuild_zebra.pl

NOTE: The scripts assume koha-functions.sh to be in /usr/share/koha/bin.
Finding a better location for this shell library may be hard.

Test plan:
Run koha-indexer for a regular package install or a dev install.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Comment 13 Owen Leonard 2016-07-13 16:16:01 UTC
Created attachment 53386 [details] [review]
[SIGNED-OFF] Bug 16733: Adjust koha-rebuild-zebra

[1] Add a call to the new adjust_paths_dev_install
[2] Differentiate location of rebuild_zebra.pl
[3] Replace a hardcoded path by $PERL5LIB

Test plan:
Adjust a biblio record in package or dev install.
Run koha-rebuild-zebra -b -z for same instance.
Verify that the change has been indexed.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Comment 14 Owen Leonard 2016-07-13 16:16:11 UTC
Created attachment 53387 [details] [review]
[SIGNED-OFF] Bug 16733: Adjust other debian scripts using PERL5LIB

This patch makes the following changes:

koha-foreach, koha-upgrade-schema (shell scripts):
[1] Read default file
[2] Include helper functions
[3] Add call to adjust_paths_dev_install
[4] Replace hardcoded path by $PERL5LIB

koha-shell (perl script):
[1] Remove hardcoded lib path
[2] Add a sub that reads PERL5LIB from default or koha-conf, just as the
    shell scripts do.

koha-plack (shell script), plack.psgi:
[1] Add call to adjust_paths_dev_install
[2] Remove hardcoded lib path
[3] Add installer path to PERL5LIB, remove it from plack.pgsi

koha-sitemap (shell script):
[1] Add call to adjust_paths_dev_install
[2] Remove hardcoded lib path
[3] Add installer path to PERL5LIB
[4] Adjust path for call to sitemap cron job

koha-start-sip (shell script):
[1] Read default file
[2] Include helper functions
[3] Add call to adjust_paths_dev_install
[4] Adjust path to C4/SIP

koha-stop-sip (shell script):
[1] Remove KOHA_CONF and PERL5LIB (not needed to stop the daemon)
[2] Same for paths in daemon client options

NOTE: Script debian/scripts/koha-upgrade-to-3.4 has been left out
intentionally.

Test plan:
[1] Regular install:
    Run koha-foreach echo Hi
    Run koha-upgrade-schema yourinstance
    Run koha-shell yourinstance
    If you have plack, run koha-plack --start|--stop yourinstance
    Run koha-sitemap --generate yourinstance
    Run koha-start-sip yourinstance
    Run koha-stop-sip yourinstance

[2] Dev install [yourinstance] with <dev_install> in koha-conf.xml:
    Run koha-upgrade-schema yourinstance
    Run koha-shell yourinstance
    If you have plack: koha-plack --start|--stop yourinstance
    Run koha-sitemap --generate yourinstance
    Run koha-start-sip yourinstance
    Run koha-stop-sip yourinstance

[3] Git grep on koha/lib
    You should no longer see occurrences in debian/scripts except:
    koha-translate: see report 16749
    koha-upgrade-to-3.4: left out intentionally

[4] Git grep on koha/bin
    You should only see hits for lines with koha-functions in the
    debian scripts except:
    koha-upgrade-to-3.4: left out intentionally

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Most scripts tested on Wheezy (although it would not matter much).
Plack script tested on Jessie.

Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Comment 15 Owen Leonard 2016-07-13 16:17:47 UTC
Tested successfully in a gitified package installation.
Comment 16 Marcel de Rooy 2016-07-14 06:26:14 UTC
(In reply to Owen Leonard from comment #15)
> Tested successfully in a gitified package installation.

Thanks!
Comment 17 Marcel de Rooy 2016-09-12 10:32:09 UTC
Created attachment 55491 [details] [review]
Bug 16733: [Follow-up] Add $home to api path too

In the meantime api was enabled in plack.psgi and needs a little tweak
too for a dev install.
Comment 18 Jonathan Druart 2016-12-02 09:56:05 UTC
Yesterday I faced the problem of using koha-translate on a gitified intall, `koha-translate --install es-ES` won't install the Spanish templates in the gitified dirs.
I remembered this bug and thought it would have fixed it, but it does not.
Is it expected?
Comment 19 Marcel de Rooy 2016-12-02 10:02:18 UTC
See dependency 16749
Comment 20 David Cook 2017-01-12 02:05:23 UTC
I like this idea, but I don't use Debian often enough to be useful in terms of testing...
Comment 21 Jonathan Druart 2017-01-18 09:48:58 UTC
Created attachment 59146 [details] [review]
Bug 16733: Add adjust_paths_dev_install to koha-functions.sh

This new function checks koha-conf.xml for a given instance and if it
contains a dev_install line, it adjusts PERL5LIB and KOHA_HOME
accordingly. Otherwise it does not touch the values of these
variables as normally read from /etc/default/koha-common.

The function will be used in various debian scripts to allow for more
flexibility with dev installs. And at the same time aiming to make better
use of PERL5LIB and KOHA_HOME.

Test plan:
[1] Add <dev_install>/not/there</dev_install> to your koha-conf.xml.
[2] Run on the command line:
        PERL5LIB=test
        source [path-to-your-instance]/debian/scripts/koha-functions.sh
        echo $PERL5LIB
        adjust_paths_dev_install [name-of-your-instance]
        echo $PERL5LIB
    The last echo should be: /not/there
[3] Remove the <dev_install> line and repeat step 2.
    The last echo should be: test

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 22 Jonathan Druart 2017-01-18 09:49:03 UTC
Created attachment 59147 [details] [review]
Bug 16733: Adjust koha-indexer

[1] Add a call to the new adjust_paths_dev_install
[2] Differentiate location of rebuild_zebra.pl

NOTE: The scripts assume koha-functions.sh to be in /usr/share/koha/bin.
Finding a better location for this shell library may be hard.

Test plan:
Run koha-indexer for a regular package install or a dev install.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 23 Jonathan Druart 2017-01-18 09:49:08 UTC
Created attachment 59148 [details] [review]
Bug 16733: Adjust koha-rebuild-zebra

[1] Add a call to the new adjust_paths_dev_install
[2] Differentiate location of rebuild_zebra.pl
[3] Replace a hardcoded path by $PERL5LIB

Test plan:
Adjust a biblio record in package or dev install.
Run koha-rebuild-zebra -b -z for same instance.
Verify that the change has been indexed.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 24 Jonathan Druart 2017-01-18 09:49:13 UTC
Created attachment 59149 [details] [review]
Bug 16733: Adjust other debian scripts using PERL5LIB

This patch makes the following changes:

koha-foreach, koha-upgrade-schema (shell scripts):
[1] Read default file
[2] Include helper functions
[3] Add call to adjust_paths_dev_install
[4] Replace hardcoded path by $PERL5LIB

koha-shell (perl script):
[1] Remove hardcoded lib path
[2] Add a sub that reads PERL5LIB from default or koha-conf, just as the
    shell scripts do.

koha-plack (shell script), plack.psgi:
[1] Add call to adjust_paths_dev_install
[2] Remove hardcoded lib path
[3] Add installer path to PERL5LIB, remove it from plack.pgsi

koha-sitemap (shell script):
[1] Add call to adjust_paths_dev_install
[2] Remove hardcoded lib path
[3] Add installer path to PERL5LIB
[4] Adjust path for call to sitemap cron job

koha-start-sip (shell script):
[1] Read default file
[2] Include helper functions
[3] Add call to adjust_paths_dev_install
[4] Adjust path to C4/SIP

koha-stop-sip (shell script):
[1] Remove KOHA_CONF and PERL5LIB (not needed to stop the daemon)
[2] Same for paths in daemon client options

NOTE: Script debian/scripts/koha-upgrade-to-3.4 has been left out
intentionally.

Test plan:
[1] Regular install:
    Run koha-foreach echo Hi
    Run koha-upgrade-schema yourinstance
    Run koha-shell yourinstance
    If you have plack, run koha-plack --start|--stop yourinstance
    Run koha-sitemap --generate yourinstance
    Run koha-start-sip yourinstance
    Run koha-stop-sip yourinstance

[2] Dev install [yourinstance] with <dev_install> in koha-conf.xml:
    Run koha-upgrade-schema yourinstance
    Run koha-shell yourinstance
    If you have plack: koha-plack --start|--stop yourinstance
    Run koha-sitemap --generate yourinstance
    Run koha-start-sip yourinstance
    Run koha-stop-sip yourinstance

[3] Git grep on koha/lib
    You should no longer see occurrences in debian/scripts except:
    koha-translate: see report 16749
    koha-upgrade-to-3.4: left out intentionally

[4] Git grep on koha/bin
    You should only see hits for lines with koha-functions in the
    debian scripts except:
    koha-upgrade-to-3.4: left out intentionally

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Most scripts tested on Wheezy (although it would not matter much).
Plack script tested on Jessie.

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 25 Jonathan Druart 2017-01-18 09:49:18 UTC
Created attachment 59150 [details] [review]
Bug 16733: [Follow-up] Add $home to api path too

In the meantime api was enabled in plack.psgi and needs a little tweak
too for a dev install.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 26 Marcel de Rooy 2017-01-18 11:14:21 UTC
Thanks Jonathan
Comment 27 Kyle M Hall 2017-01-20 14:14:30 UTC
Pushed to master for 17.05, thanks Marcel!
Comment 28 Katrin Fischer 2017-01-22 20:31:02 UTC
This won't get ported back to 16.11.x as it is an enhancement.
Comment 29 Jonathan Druart 2017-01-23 08:27:06 UTC
(In reply to Katrin Fischer from comment #28)
> This won't get ported back to 16.11.x as it is an enhancement.

It would be very useful for devs to get it backported to stable releases.
Comment 30 Marcel de Rooy 2017-01-23 08:39:17 UTC
I have no strong opinion on it, but I am using a rebased version on 3.22 in production.
Comment 31 Katrin Fischer 2017-01-23 10:14:14 UTC
Can you explain why this is helpful?
More worried about breaking something for the non-devs to be honest - devs always find a way ;)
Comment 32 Jonathan Druart 2017-01-23 10:33:09 UTC
(In reply to Katrin Fischer from comment #31)
> Can you explain why this is helpful?
> More worried about breaking something for the non-devs to be honest - devs
> always find a way ;)

Typically to avoid the issue you just got on bug 17731 :)
With these patches the debian script will execute the rebuild_zebra script from the gitified repo and not from the /usr/share/koha dir.
Comment 33 Tomás Cohen Arazi 2017-01-30 15:15:05 UTC
Guys, this patchset broke kohadevbox. It is probably an integration issue, but as dev_install is not present I would expect it to use the package's paths, and instead of that it just breaks.
Comment 34 Marcel de Rooy 2017-01-31 13:09:16 UTC
(In reply to Tomás Cohen Arazi from comment #33)
> Guys, this patchset broke kohadevbox. It is probably an integration issue,
> but as dev_install is not present I would expect it to use the package's
> paths, and instead of that it just breaks.

Yes, that should be. Will have a look too..
Comment 35 Katrin Fischer 2017-02-07 07:07:44 UTC
I have decided not to push these patches to 16.11.x as there appear to be some  issues and they touch code in a lot of places.
Comment 36 Marcel de Rooy 2017-02-09 12:11:10 UTC
Further fixes or discussion on bug 16749 please.
Comment 37 Mirko Tietgen 2017-02-13 13:19:18 UTC
Looks like this breaks koha-common.postinst. Do not push to *stable. Thanks.
Comment 38 Marcel de Rooy 2017-02-13 13:24:15 UTC
(In reply to Mirko Tietgen from comment #37)
> Looks like this breaks koha-common.postinst. Do not push to *stable. Thanks.
Enhancement :) Not for backporting. Note that bug 16749 should follow it. First two patches may solve your problem..
Comment 39 Jonathan Druart 2017-04-11 13:52:05 UTC
I really would like to see them backported, it will greatly simplify setup of installation based on stable releases and so backporting.
Comment 40 Katrin Fischer 2017-04-22 06:08:51 UTC
What about comment #37?
Comment 41 Jonathan Druart 2017-04-24 13:17:03 UTC
(In reply to Katrin Fischer from comment #40)
> What about comment #37?

Comment 38 says it is fixes on bug 16749
Comment 42 Katrin Fischer 2017-05-01 10:59:40 UTC
Waiting for bug 18502 to reconsider backporting (visible when checking the graph)
Comment 43 Katrin Fischer 2017-05-14 10:53:08 UTC
I have made an attempt to backport this, but I am running into conflicts.
- Patches from this (bug 16733) apply cleanly
- Patches from next dependency bug 16749 conflict: 
could not apply 35fad66... Bug 16749: Adjustments for koha-plack

Jonathan, if you still want this in, please help me figure this out!
Comment 44 Jonathan Druart 2017-05-15 16:16:53 UTC
I have fixed the conflict, applied all patches and pushed the branch on my github account, on the branch bug_16733-16.11.x: https://github.com/joubu/Koha/tree/bug_16733-16.11.x
Comment 45 Katrin Fischer 2017-05-16 05:57:50 UTC
I've merged these patches in from Jonathan's branch, no conflicts this time.
Comment 46 Katrin Fischer 2017-05-16 06:00:20 UTC
These patches have been pushed to 16.11.x and will be in 16.11.08.
Comment 47 Mason James 2017-05-24 03:06:12 UTC
Enhancement, skipping for 16.05.x