Bug 9316

A note, this is pretty preliminary so please TEST TEST TEST and let me know if you spot anything.  I will try and patch ASAP.

*** Bug 9100 has been marked as a duplicate of this bug. ***

A few notes so far:

I'm doing my initial testing on a comletely clean install of Debian Stable.

I'm also working through it as a dev type install

  1. /home/koha/koha-dev/var/run/plack is not created during make install, therefore the make issues the error: chmod: cannot access '/home/koha/koha-dev/var/run/plack'  - I created this directory manual an re-ran the make install.. no other errors were reported.

  2. I noticed that koha-httpd.conf is still created (along with koha-nginx.conf) under koha-dev/etc/.. should this be the case still for nginx configurations?

  3. The two scripts created under /koha-dev/bin are not created with executable permission like the rest of the scripts within that directory? (koha-plack.sh and koha.psgi)

After correcting the above simple problems and symlinking the koha-nginx.conf to /etc/nginx/site-enabled and restarting nginx.

  4. Attempting to run 'sudo /etc/init.d/koha-plack.sh start' results in the following error:

  Starting Koha Plack Daemon
  /etc/init.d/koha-plack: 72: [[: not found
  /etc/init.d/koha-plack: 72: starman: not found

At this point I stopped.

Great start on the work, but for me it's not yet working.. I'm reminding myself how plack is meant to work now and will report back if/when I get further.

Added starman with cpan now..

Re-running 'sudo /etc/init.d/koha-plack.sh start' now results in:

Starting Koha Plack Daemon
/etc/init.d/koha-plack: 72: [[: not found
chown: cannot access `/home/koha/koha-dev/var/run/plack/plack.sock': No such file or directory

Hey Martin,

You're absolutely right about the directories and permissions.  I forgot I made those early on and forgot to include them in the patch.  I'll push an update soon and make sure those directories get created with the correct permissions.  I'll also take a look at the plack init script and see what could be causing the ]] warning. 

Did you install the starman deps?  I noticed my copy of starman was in /usr/local/bin but it could be different for you. If starman isn't in your path that would explain why it isn't executing.

Thanks for testing, I had a feeling I would be running into some permission/file issues like these.

Elliott Davis

Hi Elliott,

It's great to see the PLACK stuff heading towards the mainstream with these options.. great work.. hope my comments haven't been sounding too negative.

As a clean install I hadn't actually grabbed the Starman modules yet (oops), I've now installed them using cpan and the script executes OK.. although it still gives that [[ error. (I had done used aptitude to install libplack-perl libcgi-emulate-psgi-perl libcgi-compile-perl.. but cpanm starman was also needed)

Now i've got nginx and koha-plack, but.. but navigating to either Staff or OPAC just gives a 502 Bad Gateway Error.  Not sure where I'm going wrong for this bit. I'm testing on a headless machine running nginx 0.7.67 and using dns entries. (both staff and opac on port 80).

Martin,

I am just grateful for getting some feedback.  I didn't want this to get stuck and never make it.  Don't worry I promise to take all comments constructively :)

I too had a go with the 502 errors from nginx.  What I was seeing was a permissions error for the user for the plack script. You are probably seeing the same issue.  If you run a tail -f /var/log/nginx/error.log (or where ever your log dir is) you should probably see something similar.  I assumed it was just my setup.  I resolved the issue by starting plack as the sudo user.  Zebra runs as the koha user but I am thinking it may be a requirement to run plack as root.

Let me know what you think/see. 

Elliott Davis

(In reply to comment #9)
> I too had a go with the 502 errors from nginx.  What I was seeing was a
> permissions error for the user for the plack script. You are probably seeing
> the same issue.  If you run a tail -f /var/log/nginx/error.log (or where
> ever your log dir is) you should probably see something similar.  I assumed
> it was just my setup.  I resolved the issue by starting plack as the sudo
> user.  Zebra runs as the koha user but I am thinking it may be a requirement
> to run plack as root.

I am running Plack as my regular user without any problems. My permissions are:
jcamins@kohadev:~/kohaclone$ ls -lR ~/koha-dev/var/run
/home/jcamins/koha-dev/var/run:
total 8
-rw-rw-r-- 1 jcamins jcamins     5 Jan  1 22:00 plack.pid
srwxrwxr-x 1 jcamins www-data    0 Jan  1 22:00 plack.sock
drwxr-xr-x 2 jcamins www-data 4096 Dec 28 14:31 zebradb

/home/jcamins/koha-dev/var/run/zebradb:
total 4
srw-rw-rw- 1 jcamins jcamins   0 Dec 28 14:31 authoritysocket
srw-rw-rw- 1 jcamins jcamins   0 Dec 28 14:31 bibliosocket
-rw-r--r-- 1 jcamins www-data 26 Oct  8 16:11 README

I think the important bit is the group of the plack.sock file.

That's interesting.. It was indeed a permissions thing (once I'd sorted out a few other missing dependencies that were flagged at the end of the Makefile.pl run)

I found the issue was solved by giving 'write' permission for the 'group'..

My permissions are now: 

-rw-r--r-- 1 root root        5 Jan  2 18:49 plack.pid
srwxrwxr-x 1 koha www-data    0 Jan  2 18:49 plack.sock

Although running /etc/init.d/koha-plack seems to reset ownership to koha:koha?

PS: The missing dependencies that I had neglected earlier were:

starman
CGI::Emulate::PSGI
CGI::Compile
Plack::Middleware::Debug

Further comments now I've got that bit further through.. (though these aren't blockers, they may be worth mentioning in their own right?)

1. Whilst we're in that state where the web installer shows, the OPAC just shows a 'not found' message, whilst in the non-plack implementation this defaults to the maintenance message (I think?)

2. The koha.org-logo.gif no longer displays in the web installer. (The related error appearing in /var/log/nginx/error.log is: 013/01/02 18:58:01 [error] 1192#0: *62 open() "/home/koha/kohaclone/koha-tmpl/intranet-tmpl/intranet-tmpl/prog/img/koha.org-logo.gif" failed (2: No such file or directory), client: 192.168.0.10, server: plack.koha-ptfs.eu, request: "GET /intranet-tmpl/prog/img/koha.org-logo.gif HTTP/1.1", host: "plack.koha-ptfs.eu", referrer: "http://plack.koha-ptfs.eu/cgi-bin/koha/installer/install.pl" )

...

I think issue 2. is endemic of a more serious issue. (I'm still testing on a 'dev' type install)

Now that i've completed the web installer, the intranet login apeared unstyled.. and after login the whole page appear unstyled. nginx error.log is full of 'No such file or directory' errors for js, images and css?

Found a fix for the above missing supplementary files.

Looking at the below error in /var/log/nginx/error.log:

2013/01/02 19:16:59 [error] 1192#0: *135 open() "/home/koha/kohaclone/koha-tmpl/intranet-tmpl/intranet-tmpl/prog/en/includes/favicon.ico" failed (2: No such file or directory), client: 192.168.0.10, server: plack.koha-ptfs.eu, request: "GET /intranet-tmpl/prog/en/includes/favicon.ico HTTP/1.1", host: "plack.koha-ptfs.eu"

I noted that is looked as though an extra '/intranet-tmpl/' was appearing in the directory tree.  I edited the koha-nginx.conf file removing '/opac-tmpl/' from line 18 and '/intranet-tmpl/' from line 42.

I'm not expert in nginx config file though.. so hopefully you'll spot if that's not the way it should be.  This is all on a dev install still.. I'll also test with a from scratch install using the production install mechanism.

Also.. is the plDebug bar meant to display be default in a plack install (or have I somehow inadvertently added a debug environment variable without meaning to?)

Martin

I've made the requested changes to the nginx configs and the plack startup script.  You should still have to start the plack script as root.  I am still trying to figure out how to create the correct permissions to run plack as the koha user.  Typically scripts in /etc/init.d are run as root but if I remember correctly koha-zebra-ctl.sh is run as the koha user. 

I also made one tine change for non-dev installs to path the run directory correctly.

not applied that last one yet.. but spotted an error in the code..

Your chmod in the new koha-plack.sh  You've missed the "T" at then end, and the permission required I think is 666, 775 doesn't appear to work.. (666 is the same as the zebra sockets permissions)


I'd have made the cahnge myself.. but it looks like your working on it so I thought it sensible to wait until your happy with what you've got.

As a niggle. Might it be sensible to split up the lock and run files to follow the conventions in the zebra server.. (ie. put the lock file inside /var/lock/plack)?

meant to also add.. for the same script as mentioned above.. I've never really understood the fine points of bash vs zsh, vs sh and the like.. but why use straight sh with a -e option as opposed to bash (taking again the zebra startup scripts as my initial basis for the question?)

the reason I chose a .sh was mostly due to the fact that I don't have much experience in shell scripting of any kind so I modeled the file after koha-zebra-ctl.sh.  I have altered the socket permissions to run under 660 rather that 775 or 666.  


Feel free to modify from here on out I think I am done pushing changed to it for today.  

Elliott

Martin, please make sure you have your git user set up so that you can get credit for your work:
  git config --global user.name "your NAME"
  git config --global user.email "your@mail.com"

Ooops.. Created a new VM for testing just this patch and forgot to finish up the git config. Cheers Jared.

I think I would like to see a change: rather than asking the user to decide between Apache and nginx/Plack when running Makefile.PL, I would like to see both sets of configuration files generated regardless. The user can then decide which they want to use, and switch at will.

Applying: Bug 9316: Add ability to install/configure nginx - Minor Corrections
fatal: sha1 information is lacking or useless (C4/Installer/PerlDependencies.pm).
Repository lacks necessary blobs to fall back on 3-way merge.
Cannot fall back to three-way merge.

Not sure what that means or how to resolve it.

Patches appear to be applied in the wrong order.

> 4) Run through the make file, you will notice 5 new configuration options.

No, I don't. Am I missing something obvious?

(In reply to comment #35)
> Patches appear to be applied in the wrong order.
> 
> > 4) Run through the make file, you will notice 5 new configuration options.
> 
> No, I don't. Am I missing something obvious?

They do seem to be in the wrong order in BZ.  The new configuration options are Staff/OPAC DNS and port entries.

I think the problem with the merge was the incorrect order.  If you apply the patches backwards it should be fine.

Yes i did it backwards. I have seen the DNS and port options when running the makefile, I thought there was some NGiNX option supposed to come up

> 5) Make sure you choose NGiNX as your webserver

So this is not supposed to happen when I run the makefile?

(In reply to comment #37) 
> > 5) Make sure you choose NGiNX as your webserver
> 
> So this is not supposed to happen when I run the makefile?

This was in an older version of the patch.  It was removed at the request of the RM.

I see. I did not read the whole backlog of the bug. It would be nice to have a revised version of the test plan if it does not apply anymore. Are there more changes to it that I should be aware of?

that should be the only one.  

To Test:
1) Ensure that you have nginx installed
2) Esure that you have the following packages installed libplack-perl libcgi-emulate-psgi-perl libcgi-compile-perl
3) Apply patch
4) Run through the make file, you will notice 4 new configuration options.
5) You may choose whatever ports you wish and also whatever DNS entries you wish
6) After you're done configuring run make
7) Run make install (you may need sudo if you've used sudo before)
8) Start plack (you may wish to copy the script to init.d) with the script in koha-dev/bin/koha-plack.sh (requires sudo)
9) TURN OFF APACHE, and start NGiNX
10) go to your web browser and see if everything fires up as normal.

I got 504 Gateway Time-out, then 502 Gateway Time-out.

> 2013/03/09 09:53:30 [error] 22634#0: *17 connect() to unix:/var/run/testkoha/plack/plack.sock failed (11: Resource temporarily unavailable) while connecting to upstream, client: 192.168.2.1, server: localhost, request: "GET / HTTP/1.1", upstream: "http://unix:/var/run/testkoha/plack/plack.sock:/cgi-bin/koha/mainpage.pl", host: "192.168.2.25:8090"

I meant »502 Bad Gateway«

Log message in readable form:

2013/03/09 09:53:30 [error] 22634#0: *17 connect() to unix:/var/run/testkoha/plack/plack.sock failed (11: Resource temporarily unavailable) while connecting to upstream, client: 192.168.2.1, server: localhost, request: "GET / HTTP/1.1", upstream: "http://unix:/var/run/testkoha/plack/plack.sock:/cgi-bin/koha/mainpage.pl", host: "192.168.2.25:8090"

Created attachment 21818 [details] [review]
Bug 9316: Add ability to install/configure nginx

Amended permissions for plack socket and fixed type
Altered mount location for opac to reflect current koha path settings

Update:

Resolved merge problems

Created attachment 21819 [details] [review]
Bug 9316: Add ability to install/configure nginx - Minor Corrections

This patch adds minor changes to koha-plack.sh upstart script and a few
corrections to permissions and missing directory locations to quiet errors

This patch should be applied after Elliott Davis's patch.

Test as before:
1) Ensure that you have nginx installed
2) Esure that you have the following packages installed libplack-perl libcgi-emulate-psgi-perl libcgi-compile-perl starman
3) Apply patch
4) Run through the make file, you will notice 5 new configuration options.
5) Make sure you choose NGiNX as your webserver
6) You may choose whatever ports you wish and also whatever DNS entries you wish
7) After you're done configuring run make
8) Run make install (you may need sudo if you've used sudo before)
9) Start plack (you may wish to copy the script to init.d) with the script in koha-dev/bin/koha-plack.sh (requires sudo)
10) TURN OFF APACHE, and start NGiNX
11) go to your web browser and see if everything fires up as normal.

removed dependencies and updated init script to allow for the use of starman or plackup

fixed debian control
added a warning for nginx configurations
modified koha.psgi to enable debugging only if plack::middleware::debug had been installed

This took quite a bit of rebasing, I ended up needing to use the old git apply PATCH --reject trick

The »502 Bad Gateway« error that Mirko Tietgen is coming across is caused by  not having Plack::Middleware::Debug installed. Can this be made to be a more graceful error, or even better not be a requirement of the scripts at all? 

I see the middle as being very useful for developers, but not a requirement for everyday running.. Correct me if I'm wrong however?

Once that's fixed, I'm happy to sign off.

Pants, just spotted that I created the small follow-up patch..  Can't really sign off then.  I'll bully someone at kohacon13 hackfest if the change is made quick enough.

Patch applied cleanly, go forth and signoff

OK.. I've been trying to narrow down the bug with requiring Plack::Middleware::Debug.

I've narrowed it down to the following lines in koha.psgi:

    enable_if {
            $ENV{'KOHA_DEBUG'}
    } 'Debug',  panels => [[ 'DBITrace', level => 1 ], @$panels ];

I think the logic here is flawed somehow.  First up I think it should actually be:

    enable_if {
             $_[0]->{'KOHA_DEBUG'}
    } 'Debug',  panels => [[ 'DBITrace', level => 1 ], @$panels ];

But this also doesn't work. I think the builder checks for the module availability even if it's not actually to be loaded as part of the enable_if.

I'm baffled.. anyone got any idea's how to fix this?

Created attachment 22767 [details] [review]
Bug 9316: Add ability to install/configure nginx - Followup

Added ability to run plack without Plack::Middleware::Debug installed

I worked through it and got koha.psgi into a state where Plack::Middleware::Debug is no longer required..

I am a little confused as to how to enable KOHA_DEBUG using nginx however.. any hints would be greatly appreciated.

This should probably be a separate bug.. but upon first run, the opac displays errors instead of the maintenance page (that's with debug enabled or disabled)

Can't use an undefined value as an ARRAY reference at /usr/local/lib/perl/5.14.2/DBI.pm line 2064.
 at /home/koha/kohaclone/opac/maintenance.pl line 54

I'm a bit confused by the relationshio between this bug and bug 7844. 

Culd we say that 7844 is for developers and this bug for production? Or do they overlap?

As magnus has highlighted, bugs in this area (Plack and other infrastructure decisions) are quite fragmented.  I hope to build a dependancy tree of bugs related to this over the coming days to more clearly specify how the bugs all relate.

What's the status with this bug report?

With the plack in production as of 3.22.0 this bug needs a rethink, the patches as they stand will duplicate a lot of stuff

*** Bug 11952 has been marked as a duplicate of this bug. ***

These patches are super outdated now, and I don't think there are any serious plans to add support for Nginx with koha-common, although I think that BibLibre use Nginx at some level in front of Koha. 

If someone wants to seriously add Nginx support, they can either re-open this bug, or create a new one.