Bug 9316 - Add Nginx install options with plack
Summary: Add Nginx install options with plack
Status: In Discussion
Alias: None
Product: Koha
Classification: Unclassified
Component: Installation and upgrade (command-line installer) (show other bugs)
Version: master
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Kyle M Hall
QA Contact:
URL:
Keywords:
: 9100 (view as bug list)
Depends on:
Blocks:
 
Reported: 2012-12-22 22:30 UTC by Elliott Davis
Modified: 2016-01-04 03:18 UTC (History)
9 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Large patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
Bug 9316: Add ability to install/configure nginx (14.80 KB, patch)
2012-12-24 23:50 UTC, Elliott Davis
Details | Diff | Splinter Review
Bug 9316: Add ability to install/configure nginx (14.80 KB, patch)
2012-12-28 21:01 UTC, Elliott Davis
Details | Diff | Splinter Review
Bug 9316: Add ability to install/configure nginx (32.19 KB, patch)
2013-01-02 20:45 UTC, Elliott Davis
Details | Diff | Splinter Review
Bug 9316: Add ability to install/configure nginx (31.47 KB, patch)
2013-01-02 20:50 UTC, Elliott Davis
Details | Diff | Splinter Review
Bug 9316: Add ability to install/configure nginx (31.52 KB, patch)
2013-01-02 21:00 UTC, Elliott Davis
Details | Diff | Splinter Review
Bug 9316: Add ability to install/configure nginx (31.52 KB, patch)
2013-01-02 21:12 UTC, Elliott Davis
Details | Diff | Splinter Review
Bug 9316: Add ability to install/configure nginx (31.53 KB, patch)
2013-01-02 21:32 UTC, Elliott Davis
Details | Diff | Splinter Review
Bug 9316: Add ability to install/configure nginx (31.53 KB, patch)
2013-01-02 21:48 UTC, Elliott Davis
Details | Diff | Splinter Review
Bug 9316: Add ability to install/configure nginx (18.33 KB, patch)
2013-01-02 22:08 UTC, Elliott Davis
Details | Diff | Splinter Review
Bug 9316: Add ability to install/configure nginx (18.37 KB, patch)
2013-01-02 22:27 UTC, Elliott Davis
Details | Diff | Splinter Review
Followup patch with suggested additions (9.41 KB, patch)
2013-01-03 01:26 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 9316: Followup patch with suggested additions (9.44 KB, patch)
2013-01-03 09:24 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 9316: Add ability to install/configure nginx (17.09 KB, patch)
2013-01-23 16:06 UTC, Elliott Davis
Details | Diff | Splinter Review
Bug 9316: Add ability to install/configure nginx - Minor Corrections (15.39 KB, patch)
2013-02-07 21:45 UTC, Elliott Davis
Details | Diff | Splinter Review
Bug 9316: Add ability to install/configure nginx (16.61 KB, patch)
2013-03-04 18:13 UTC, Elliott Davis
Details | Diff | Splinter Review
Bug 9316: Add ability to install/configure nginx (14.49 KB, patch)
2013-10-05 02:08 UTC, Chris Cormack
Details | Diff | Splinter Review
Bug 9316: Add ability to install/configure nginx - Minor Corrections (13.13 KB, patch)
2013-10-05 02:08 UTC, Chris Cormack
Details | Diff | Splinter Review
Bug 9316: Add ability to install/configure nginx - Followup (1.76 KB, patch)
2013-11-07 12:18 UTC, Martin Renvoize
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Elliott Davis 2012-12-22 22:30:53 UTC
Add the ability to specify NGiNX and PLACK in the installer.
Comment 1 Elliott Davis 2012-12-24 23:50:07 UTC Comment hidden (obsolete)
Comment 2 Elliott Davis 2012-12-24 23:50:49 UTC
A note, this is pretty preliminary so please TEST TEST TEST and let me know if you spot anything.  I will try and patch ASAP.
Comment 3 Elliott Davis 2012-12-28 21:01:06 UTC Comment hidden (obsolete)
Comment 4 Jared Camins-Esakov 2012-12-30 23:45:07 UTC
*** Bug 9100 has been marked as a duplicate of this bug. ***
Comment 5 Martin Renvoize 2013-01-02 17:42:19 UTC
A few notes so far:

I'm doing my initial testing on a comletely clean install of Debian Stable.

I'm also working through it as a dev type install

  1. /home/koha/koha-dev/var/run/plack is not created during make install, therefore the make issues the error: chmod: cannot access '/home/koha/koha-dev/var/run/plack'  - I created this directory manual an re-ran the make install.. no other errors were reported.

  2. I noticed that koha-httpd.conf is still created (along with koha-nginx.conf) under koha-dev/etc/.. should this be the case still for nginx configurations?

  3. The two scripts created under /koha-dev/bin are not created with executable permission like the rest of the scripts within that directory? (koha-plack.sh and koha.psgi)

After correcting the above simple problems and symlinking the koha-nginx.conf to /etc/nginx/site-enabled and restarting nginx.

  4. Attempting to run 'sudo /etc/init.d/koha-plack.sh start' results in the following error:

  Starting Koha Plack Daemon
  /etc/init.d/koha-plack: 72: [[: not found
  /etc/init.d/koha-plack: 72: starman: not found

At this point I stopped.

Great start on the work, but for me it's not yet working.. I'm reminding myself how plack is meant to work now and will report back if/when I get further.
Comment 6 Martin Renvoize 2013-01-02 17:47:05 UTC
Added starman with cpan now..

Re-running 'sudo /etc/init.d/koha-plack.sh start' now results in:

Starting Koha Plack Daemon
/etc/init.d/koha-plack: 72: [[: not found
chown: cannot access `/home/koha/koha-dev/var/run/plack/plack.sock': No such file or directory
Comment 7 Elliott Davis 2013-01-02 17:50:25 UTC
Hey Martin,

You're absolutely right about the directories and permissions.  I forgot I made those early on and forgot to include them in the patch.  I'll push an update soon and make sure those directories get created with the correct permissions.  I'll also take a look at the plack init script and see what could be causing the ]] warning. 

Did you install the starman deps?  I noticed my copy of starman was in /usr/local/bin but it could be different for you. If starman isn't in your path that would explain why it isn't executing.

Thanks for testing, I had a feeling I would be running into some permission/file issues like these.

Elliott Davis
Comment 8 Martin Renvoize 2013-01-02 18:19:31 UTC
Hi Elliott,

It's great to see the PLACK stuff heading towards the mainstream with these options.. great work.. hope my comments haven't been sounding too negative.

As a clean install I hadn't actually grabbed the Starman modules yet (oops), I've now installed them using cpan and the script executes OK.. although it still gives that [[ error. (I had done used aptitude to install libplack-perl libcgi-emulate-psgi-perl libcgi-compile-perl.. but cpanm starman was also needed)

Now i've got nginx and koha-plack, but.. but navigating to either Staff or OPAC just gives a 502 Bad Gateway Error.  Not sure where I'm going wrong for this bit. I'm testing on a headless machine running nginx 0.7.67 and using dns entries. (both staff and opac on port 80).
Comment 9 Elliott Davis 2013-01-02 18:24:31 UTC
Martin,

I am just grateful for getting some feedback.  I didn't want this to get stuck and never make it.  Don't worry I promise to take all comments constructively :)

I too had a go with the 502 errors from nginx.  What I was seeing was a permissions error for the user for the plack script. You are probably seeing the same issue.  If you run a tail -f /var/log/nginx/error.log (or where ever your log dir is) you should probably see something similar.  I assumed it was just my setup.  I resolved the issue by starting plack as the sudo user.  Zebra runs as the koha user but I am thinking it may be a requirement to run plack as root.

Let me know what you think/see. 

Elliott Davis
Comment 10 Jared Camins-Esakov 2013-01-02 18:32:42 UTC
(In reply to comment #9)
> I too had a go with the 502 errors from nginx.  What I was seeing was a
> permissions error for the user for the plack script. You are probably seeing
> the same issue.  If you run a tail -f /var/log/nginx/error.log (or where
> ever your log dir is) you should probably see something similar.  I assumed
> it was just my setup.  I resolved the issue by starting plack as the sudo
> user.  Zebra runs as the koha user but I am thinking it may be a requirement
> to run plack as root.

I am running Plack as my regular user without any problems. My permissions are:
jcamins@kohadev:~/kohaclone$ ls -lR ~/koha-dev/var/run
/home/jcamins/koha-dev/var/run:
total 8
-rw-rw-r-- 1 jcamins jcamins     5 Jan  1 22:00 plack.pid
srwxrwxr-x 1 jcamins www-data    0 Jan  1 22:00 plack.sock
drwxr-xr-x 2 jcamins www-data 4096 Dec 28 14:31 zebradb

/home/jcamins/koha-dev/var/run/zebradb:
total 4
srw-rw-rw- 1 jcamins jcamins   0 Dec 28 14:31 authoritysocket
srw-rw-rw- 1 jcamins jcamins   0 Dec 28 14:31 bibliosocket
-rw-r--r-- 1 jcamins www-data 26 Oct  8 16:11 README

I think the important bit is the group of the plack.sock file.
Comment 11 Martin Renvoize 2013-01-02 18:55:49 UTC
That's interesting.. It was indeed a permissions thing (once I'd sorted out a few other missing dependencies that were flagged at the end of the Makefile.pl run)

I found the issue was solved by giving 'write' permission for the 'group'..

My permissions are now: 

-rw-r--r-- 1 root root        5 Jan  2 18:49 plack.pid
srwxrwxr-x 1 koha www-data    0 Jan  2 18:49 plack.sock

Although running /etc/init.d/koha-plack seems to reset ownership to koha:koha?

PS: The missing dependencies that I had neglected earlier were:

starman
CGI::Emulate::PSGI
CGI::Compile
Plack::Middleware::Debug
Comment 12 Martin Renvoize 2013-01-02 19:14:33 UTC
Further comments now I've got that bit further through.. (though these aren't blockers, they may be worth mentioning in their own right?)

1. Whilst we're in that state where the web installer shows, the OPAC just shows a 'not found' message, whilst in the non-plack implementation this defaults to the maintenance message (I think?)

2. The koha.org-logo.gif no longer displays in the web installer. (The related error appearing in /var/log/nginx/error.log is: 013/01/02 18:58:01 [error] 1192#0: *62 open() "/home/koha/kohaclone/koha-tmpl/intranet-tmpl/intranet-tmpl/prog/img/koha.org-logo.gif" failed (2: No such file or directory), client: 192.168.0.10, server: plack.koha-ptfs.eu, request: "GET /intranet-tmpl/prog/img/koha.org-logo.gif HTTP/1.1", host: "plack.koha-ptfs.eu", referrer: "http://plack.koha-ptfs.eu/cgi-bin/koha/installer/install.pl" )

...

I think issue 2. is endemic of a more serious issue. (I'm still testing on a 'dev' type install)

Now that i've completed the web installer, the intranet login apeared unstyled.. and after login the whole page appear unstyled. nginx error.log is full of 'No such file or directory' errors for js, images and css?
Comment 13 Martin Renvoize 2013-01-02 19:27:03 UTC
Found a fix for the above missing supplementary files.

Looking at the below error in /var/log/nginx/error.log:

2013/01/02 19:16:59 [error] 1192#0: *135 open() "/home/koha/kohaclone/koha-tmpl/intranet-tmpl/intranet-tmpl/prog/en/includes/favicon.ico" failed (2: No such file or directory), client: 192.168.0.10, server: plack.koha-ptfs.eu, request: "GET /intranet-tmpl/prog/en/includes/favicon.ico HTTP/1.1", host: "plack.koha-ptfs.eu"

I noted that is looked as though an extra '/intranet-tmpl/' was appearing in the directory tree.  I edited the koha-nginx.conf file removing '/opac-tmpl/' from line 18 and '/intranet-tmpl/' from line 42.

I'm not expert in nginx config file though.. so hopefully you'll spot if that's not the way it should be.  This is all on a dev install still.. I'll also test with a from scratch install using the production install mechanism.

Also.. is the plDebug bar meant to display be default in a plack install (or have I somehow inadvertently added a debug environment variable without meaning to?)

Martin
Comment 14 Elliott Davis 2013-01-02 20:45:10 UTC Comment hidden (obsolete)
Comment 15 Elliott Davis 2013-01-02 20:48:06 UTC
I've made the requested changes to the nginx configs and the plack startup script.  You should still have to start the plack script as root.  I am still trying to figure out how to create the correct permissions to run plack as the koha user.  Typically scripts in /etc/init.d are run as root but if I remember correctly koha-zebra-ctl.sh is run as the koha user. 

I also made one tine change for non-dev installs to path the run directory correctly.
Comment 16 Elliott Davis 2013-01-02 20:50:05 UTC Comment hidden (obsolete)
Comment 17 Elliott Davis 2013-01-02 21:00:38 UTC Comment hidden (obsolete)
Comment 18 Elliott Davis 2013-01-02 21:12:42 UTC Comment hidden (obsolete)
Comment 19 Elliott Davis 2013-01-02 21:32:15 UTC Comment hidden (obsolete)
Comment 20 Elliott Davis 2013-01-02 21:48:25 UTC Comment hidden (obsolete)
Comment 21 Martin Renvoize 2013-01-02 21:54:19 UTC
not applied that last one yet.. but spotted an error in the code..

Your chmod in the new koha-plack.sh  You've missed the "T" at then end, and the permission required I think is 666, 775 doesn't appear to work.. (666 is the same as the zebra sockets permissions)


I'd have made the cahnge myself.. but it looks like your working on it so I thought it sensible to wait until your happy with what you've got.

As a niggle. Might it be sensible to split up the lock and run files to follow the conventions in the zebra server.. (ie. put the lock file inside /var/lock/plack)?
Comment 22 Martin Renvoize 2013-01-02 21:56:17 UTC
meant to also add.. for the same script as mentioned above.. I've never really understood the fine points of bash vs zsh, vs sh and the like.. but why use straight sh with a -e option as opposed to bash (taking again the zebra startup scripts as my initial basis for the question?)
Comment 23 Elliott Davis 2013-01-02 22:08:44 UTC Comment hidden (obsolete)
Comment 24 Elliott Davis 2013-01-02 22:27:57 UTC Comment hidden (obsolete)
Comment 25 Elliott Davis 2013-01-02 22:30:53 UTC
the reason I chose a .sh was mostly due to the fact that I don't have much experience in shell scripting of any kind so I modeled the file after koha-zebra-ctl.sh.  I have altered the socket permissions to run under 660 rather that 775 or 666.  


Feel free to modify from here on out I think I am done pushing changed to it for today.  

Elliott
Comment 26 Martin Renvoize 2013-01-03 01:26:12 UTC Comment hidden (obsolete)
Comment 27 Jared Camins-Esakov 2013-01-03 01:28:37 UTC
Martin, please make sure you have your git user set up so that you can get credit for your work:
  git config --global user.name "your NAME"
  git config --global user.email "your@mail.com"
Comment 28 Martin Renvoize 2013-01-03 09:17:47 UTC
Ooops.. Created a new VM for testing just this patch and forgot to finish up the git config. Cheers Jared.
Comment 29 Martin Renvoize 2013-01-03 09:24:54 UTC Comment hidden (obsolete)
Comment 30 Jared Camins-Esakov 2013-01-07 20:24:00 UTC
I think I would like to see a change: rather than asking the user to decide between Apache and nginx/Plack when running Makefile.PL, I would like to see both sets of configuration files generated regardless. The user can then decide which they want to use, and switch at will.
Comment 31 Elliott Davis 2013-01-23 16:06:23 UTC Comment hidden (obsolete)
Comment 32 Elliott Davis 2013-02-07 21:45:25 UTC Comment hidden (obsolete)
Comment 33 Mirko Tietgen 2013-02-09 18:02:52 UTC
Applying: Bug 9316: Add ability to install/configure nginx - Minor Corrections
fatal: sha1 information is lacking or useless (C4/Installer/PerlDependencies.pm).
Repository lacks necessary blobs to fall back on 3-way merge.
Cannot fall back to three-way merge.

Not sure what that means or how to resolve it.
Comment 34 Elliott Davis 2013-03-04 18:13:37 UTC Comment hidden (obsolete)
Comment 35 Mirko Tietgen 2013-03-08 08:40:38 UTC
Patches appear to be applied in the wrong order.

> 4) Run through the make file, you will notice 5 new configuration options.

No, I don't. Am I missing something obvious?
Comment 36 Elliott Davis 2013-03-08 14:41:06 UTC
(In reply to comment #35)
> Patches appear to be applied in the wrong order.
> 
> > 4) Run through the make file, you will notice 5 new configuration options.
> 
> No, I don't. Am I missing something obvious?

They do seem to be in the wrong order in BZ.  The new configuration options are Staff/OPAC DNS and port entries.

I think the problem with the merge was the incorrect order.  If you apply the patches backwards it should be fine.
Comment 37 Mirko Tietgen 2013-03-08 14:53:55 UTC
Yes i did it backwards. I have seen the DNS and port options when running the makefile, I thought there was some NGiNX option supposed to come up

> 5) Make sure you choose NGiNX as your webserver

So this is not supposed to happen when I run the makefile?
Comment 38 Elliott Davis 2013-03-08 14:55:06 UTC
(In reply to comment #37) 
> > 5) Make sure you choose NGiNX as your webserver
> 
> So this is not supposed to happen when I run the makefile?

This was in an older version of the patch.  It was removed at the request of the RM.
Comment 39 Mirko Tietgen 2013-03-08 15:02:49 UTC
I see. I did not read the whole backlog of the bug. It would be nice to have a revised version of the test plan if it does not apply anymore. Are there more changes to it that I should be aware of?
Comment 40 Elliott Davis 2013-03-08 15:07:07 UTC
that should be the only one.  

To Test:
1) Ensure that you have nginx installed
2) Esure that you have the following packages installed libplack-perl libcgi-emulate-psgi-perl libcgi-compile-perl
3) Apply patch
4) Run through the make file, you will notice 4 new configuration options.
5) You may choose whatever ports you wish and also whatever DNS entries you wish
6) After you're done configuring run make
7) Run make install (you may need sudo if you've used sudo before)
8) Start plack (you may wish to copy the script to init.d) with the script in koha-dev/bin/koha-plack.sh (requires sudo)
9) TURN OFF APACHE, and start NGiNX
10) go to your web browser and see if everything fires up as normal.
Comment 41 Mirko Tietgen 2013-03-09 08:55:16 UTC
I got 504 Gateway Time-out, then 502 Gateway Time-out.

> 2013/03/09 09:53:30 [error] 22634#0: *17 connect() to unix:/var/run/testkoha/plack/plack.sock failed (11: Resource temporarily unavailable) while connecting to upstream, client: 192.168.2.1, server: localhost, request: "GET / HTTP/1.1", upstream: "http://unix:/var/run/testkoha/plack/plack.sock:/cgi-bin/koha/mainpage.pl", host: "192.168.2.25:8090"
Comment 42 Mirko Tietgen 2013-03-09 08:57:45 UTC
I meant »502 Bad Gateway«

Log message in readable form:

2013/03/09 09:53:30 [error] 22634#0: *17 connect() to unix:/var/run/testkoha/plack/plack.sock failed (11: Resource temporarily unavailable) while connecting to upstream, client: 192.168.2.1, server: localhost, request: "GET / HTTP/1.1", upstream: "http://unix:/var/run/testkoha/plack/plack.sock:/cgi-bin/koha/mainpage.pl", host: "192.168.2.25:8090"
Comment 43 Chris Cormack 2013-10-05 02:08:12 UTC
Created attachment 21818 [details] [review]
Bug 9316: Add ability to install/configure nginx

Amended permissions for plack socket and fixed type
Altered mount location for opac to reflect current koha path settings

Update:

Resolved merge problems
Comment 44 Chris Cormack 2013-10-05 02:08:44 UTC
Created attachment 21819 [details] [review]
Bug 9316: Add ability to install/configure nginx - Minor Corrections

This patch adds minor changes to koha-plack.sh upstart script and a few
corrections to permissions and missing directory locations to quiet errors

This patch should be applied after Elliott Davis's patch.

Test as before:
1) Ensure that you have nginx installed
2) Esure that you have the following packages installed libplack-perl libcgi-emulate-psgi-perl libcgi-compile-perl starman
3) Apply patch
4) Run through the make file, you will notice 5 new configuration options.
5) Make sure you choose NGiNX as your webserver
6) You may choose whatever ports you wish and also whatever DNS entries you wish
7) After you're done configuring run make
8) Run make install (you may need sudo if you've used sudo before)
9) Start plack (you may wish to copy the script to init.d) with the script in koha-dev/bin/koha-plack.sh (requires sudo)
10) TURN OFF APACHE, and start NGiNX
11) go to your web browser and see if everything fires up as normal.

removed dependencies and updated init script to allow for the use of starman or plackup

fixed debian control
added a warning for nginx configurations
modified koha.psgi to enable debugging only if plack::middleware::debug had been installed
Comment 45 Chris Cormack 2013-10-05 02:09:49 UTC
This took quite a bit of rebasing, I ended up needing to use the old git apply PATCH --reject trick
Comment 46 Martin Renvoize 2013-10-21 13:40:34 UTC
The »502 Bad Gateway« error that Mirko Tietgen is coming across is caused by  not having Plack::Middleware::Debug installed. Can this be made to be a more graceful error, or even better not be a requirement of the scripts at all? 

I see the middle as being very useful for developers, but not a requirement for everyday running.. Correct me if I'm wrong however?

Once that's fixed, I'm happy to sign off.
Comment 47 Martin Renvoize 2013-10-21 13:42:28 UTC
Pants, just spotted that I created the small follow-up patch..  Can't really sign off then.  I'll bully someone at kohacon13 hackfest if the change is made quick enough.
Comment 48 I'm just a bot 2013-10-30 07:41:18 UTC
Patch applied cleanly, go forth and signoff
Comment 49 Martin Renvoize 2013-11-07 11:11:12 UTC
OK.. I've been trying to narrow down the bug with requiring Plack::Middleware::Debug.

I've narrowed it down to the following lines in koha.psgi:

    enable_if {
            $ENV{'KOHA_DEBUG'}
    } 'Debug',  panels => [[ 'DBITrace', level => 1 ], @$panels ];

I think the logic here is flawed somehow.  First up I think it should actually be:

    enable_if {
             $_[0]->{'KOHA_DEBUG'}
    } 'Debug',  panels => [[ 'DBITrace', level => 1 ], @$panels ];

But this also doesn't work. I think the builder checks for the module availability even if it's not actually to be loaded as part of the enable_if.

I'm baffled.. anyone got any idea's how to fix this?
Comment 50 Martin Renvoize 2013-11-07 12:18:36 UTC
Created attachment 22767 [details] [review]
Bug 9316: Add ability to install/configure nginx - Followup

Added ability to run plack without Plack::Middleware::Debug installed
Comment 51 Martin Renvoize 2013-11-07 12:20:28 UTC
I worked through it and got koha.psgi into a state where Plack::Middleware::Debug is no longer required..

I am a little confused as to how to enable KOHA_DEBUG using nginx however.. any hints would be greatly appreciated.
Comment 52 Martin Renvoize 2013-11-08 09:34:39 UTC
This should probably be a separate bug.. but upon first run, the opac displays errors instead of the maintenance page (that's with debug enabled or disabled)

Can't use an undefined value as an ARRAY reference at /usr/local/lib/perl/5.14.2/DBI.pm line 2064.
 at /home/koha/kohaclone/opac/maintenance.pl line 54
Comment 53 Magnus Enger 2014-03-12 10:52:12 UTC
I'm a bit confused by the relationshio between this bug and bug 7844. 

Culd we say that 7844 is for developers and this bug for production? Or do they overlap?
Comment 54 Martin Renvoize 2014-03-12 14:36:22 UTC
As magnus has highlighted, bugs in this area (Plack and other infrastructure decisions) are quite fragmented.  I hope to build a dependancy tree of bugs related to this over the coming days to more clearly specify how the bugs all relate.
Comment 55 Joonas Kylmälä 2015-08-31 09:12:52 UTC
What's the status with this bug report?
Comment 56 Chris Cormack 2016-01-04 03:18:15 UTC
With the plack in production as of 3.22.0 this bug needs a rethink, the patches as they stand will duplicate a lot of stuff