Bug 15714 - Remove zebra.log from debian scripts and add optional log levels
Summary: Remove zebra.log from debian scripts and add optional log levels
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Packaging (show other bugs)
Version: Main
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Marcel de Rooy
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-02-01 14:34 UTC by Marcel de Rooy
Modified: 2017-12-07 22:16 UTC (History)
6 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Small patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
Bug 15714: Add get_loglevels to koha-functions.sh (1.47 KB, patch)
2016-02-03 15:06 UTC, Marcel de Rooy
Details | Diff | Splinter Review
Bug 15714: Remove zebra.log and add loglevels (3.98 KB, patch)
2016-02-03 15:06 UTC, Marcel de Rooy
Details | Diff | Splinter Review
Bug 15714: Add zebra_loglevels to koha-conf.xml (1.37 KB, patch)
2016-02-03 15:06 UTC, Marcel de Rooy
Details | Diff | Splinter Review
Bug 15714: Add zebra_loglevels to koha-conf.xml (1.52 KB, patch)
2016-02-15 07:50 UTC, Marcel de Rooy
Details | Diff | Splinter Review
[SIGNED-OFF] Bug 15714: Add get_loglevels to koha-functions.sh (1.50 KB, patch)
2016-02-16 03:39 UTC, Mark Tompsett
Details | Diff | Splinter Review
[SIGNED-OFF] Bug 15714: Remove zebra.log and add loglevels (4.00 KB, patch)
2016-02-16 03:39 UTC, Mark Tompsett
Details | Diff | Splinter Review
[SIGNED-OFF] Bug 15714: Add zebra_loglevels to koha-conf.xml (1.54 KB, patch)
2016-02-16 03:39 UTC, Mark Tompsett
Details | Diff | Splinter Review
Bug 15714: Add get_loglevels to koha-functions.sh (1.55 KB, patch)
2016-02-16 10:30 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 15714: Remove zebra.log and add loglevels (4.06 KB, patch)
2016-02-16 10:30 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 15714: Add zebra_loglevels to koha-conf.xml (1.60 KB, patch)
2016-02-16 10:30 UTC, Jonathan Druart
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Marcel de Rooy 2016-02-01 14:34:58 UTC
Scripts koha-start-zebra (as well as koha-stop-zebra) refer to zebra.log.
But they also use zebra-output.log and zebra-error.log.

The scripts do not correctly use the stdout, stderr and output options from daemon. Either you use output (for both) or you use stdout and stderr.

Patch forthcoming.
Will also make it possible to add log level "log" to zebrasrv -v to include requests in the zebra-output.log (optionally via koha-conf).
Comment 1 David Cook 2016-02-02 03:46:30 UTC
Sounds like a great idea. I noticed a little while ago that the Zebra log settings weren't optimal on Debian, so I'd be happy to test something. 

Especially happy about the added log level, as that will make it easier to troubleshoot Zebra on Debian...
Comment 2 Marcel de Rooy 2016-02-03 15:06:06 UTC Comment hidden (obsolete)
Comment 3 Marcel de Rooy 2016-02-03 15:06:10 UTC Comment hidden (obsolete)
Comment 4 Marcel de Rooy 2016-02-03 15:06:14 UTC Comment hidden (obsolete)
Comment 5 Mark Tompsett 2016-02-08 22:22:35 UTC
Comment on attachment 47599 [details] [review]
Bug 15714: Add zebra_loglevels to koha-conf.xml

Review of attachment 47599 [details] [review]:
-----------------------------------------------------------------

::: debian/templates/koha-conf-site.xml.in
@@ +284,5 @@
>   <use_zebra_facets>1</use_zebra_facets>
>   <queryparser_config>/etc/koha/searchengine/queryparser.yaml</queryparser_config>
>   <log4perl_conf>__KOHA_CONF_DIR__/log4perl.conf</log4perl_conf>
> + <!-- uncomment the following line if you need different zebra log levels -->
> + <!-- <zebra_loglevels>fatal,warn</zebra_loglevels> -->

I think this should be none,fatal,warn.
Comment 6 Marcel de Rooy 2016-02-09 12:56:18 UTC
(In reply to M. Tompsett from comment #5)
> Comment on attachment 47599 [details] [review] [review]
> > + <!-- uncomment the following line if you need different zebra log levels -->
> > + <!-- <zebra_loglevels>fatal,warn</zebra_loglevels> -->
> 
> I think this should be none,fatal,warn.

Mark, thx for looking into this.
You are right that none, fatal, warn is the default.
But I explicitly did not put that default in the commented line; that would not change anything. As the comment mentions, you need to uncomment it to get different(!) log levels. We may assume that someone editing koha-conf knowns that changing config files has side effects :)
Comment 7 Mark Tompsett 2016-02-12 23:13:45 UTC
(In reply to Marcel de Rooy from comment #6)
> (In reply to M. Tompsett from comment #5)
> > I think this should be none,fatal,warn.
> 
> But I explicitly did not put that default in the commented line; that would
> not change anything.

In its current state, yes.


> As the comment mentions, you need to uncomment it to
> get different(!) log levels. We may assume that someone editing koha-conf
> knowns that changing config files has side effects :)

There is the possibility that they may think fatal,warn is the default to change. Worse yet, they may not be aware of none. How about something like:

<!-- If you need different zebra log levels,
     uncomment the following comment.
     Valid levels include: none, fatal, and warn.
     List them in any comma-separated list order desired. -->
<!-- <zebra_loglevels>fatal,warn</zebra_loglevels> -->
Comment 8 Marcel de Rooy 2016-02-15 07:50:42 UTC Comment hidden (obsolete)
Comment 9 Marcel de Rooy 2016-02-15 07:51:22 UTC
(In reply to M. Tompsett from comment #7)
> There is the possibility that they may think fatal,warn is the default to
> change. Worse yet, they may not be aware of none. How about something like:

Thx. Adjusted accordingly.
Comment 10 Mark Tompsett 2016-02-16 03:38:39 UTC
Here is what I did to test this:
Create an empty VM (e.g. an Debian 8.3 VM)
Roll a distribution off master+15714
Install it on that VM.
sudo koha-stop-zebra library
ps aux | grep zebra
sudo koha-start-zebra library
ps aux | grep zebra
--- These should be defaults
sudo vi /etc/koha/site/library/koha-conf.xml
--- tweak the zebra log levels
sudo koha-stop-zebra library
ps aux | grep zebra
sudo koha-start-zebra library
ps aux | grep zebra
--- These should match what was tweaked.
Comment 11 Mark Tompsett 2016-02-16 03:39:08 UTC Comment hidden (obsolete)
Comment 12 Mark Tompsett 2016-02-16 03:39:16 UTC Comment hidden (obsolete)
Comment 13 Mark Tompsett 2016-02-16 03:39:23 UTC Comment hidden (obsolete)
Comment 14 Mark Tompsett 2016-02-16 03:41:03 UTC
See how I test in comment #10. I suppose putting in searchable data and re-indexing and making sure it searches adequately should also be done, but given that it starts and stops properly, I have no reason to doubt it would suddenly break that way.
Comment 15 Marcel de Rooy 2016-02-16 07:28:39 UTC
(In reply to M. Tompsett from comment #14)
> See how I test in comment #10. I suppose putting in searchable data and
> re-indexing and making sure it searches adequately should also be done, but
> given that it starts and stops properly, I have no reason to doubt it would
> suddenly break that way.

Thx
Comment 16 Jonathan Druart 2016-02-16 10:30:33 UTC
Created attachment 48103 [details] [review]
Bug 15714: Add get_loglevels to koha-functions.sh

Preparing for including different log levels in koha-start-zebra.
Note that the default response is none,fatal,warn; this is equal
to the options passed to zebrasrv in koha-start-zebra (for now).

Test plan:
[1] Run on the command line: source koha-functions.sh
[2] Type get_loglevels [your_instance]
    By default, you should see: none,fatal,warn
[3] Add this line to your koha-conf:
    <zebra_loglevels>how,are,you</zebra_loglevels>
[4] Run get_loglevels [your_instance] again.
    Remove the nonsense log levels again..

Signed-off-by: Mark Tompsett <mtompset@hotmail.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 17 Jonathan Druart 2016-02-16 10:30:36 UTC
Created attachment 48104 [details] [review]
Bug 15714: Remove zebra.log and add loglevels

The file zebra.log is actually not used, because daemon's --output
parameter sends all message to zebra-output (making that stdout
goes there too and --stdout does nothing). We do not need any
differentiation (if any at all) here; behavior does not change.

Note: zebra-error will only contain the daemon's error messages and not
the zebra error messages! So you will probably only find messages about
stopping zebra in zebra-error.

The loglevels are by default none,fatal,warn and can be changed via the
zebra_loglevels config variable (read by koha-functions.sh). If you remove
'none', you will have request-messages in the log. You can also keep 'none'
and add 'request' to achieve something similar (undocumented however).

Some of the parameters passed to daemon when stopping zebra are
superfluous and have been removed.

Test plan:
[1] Remove the file zebra.log. Look at last lines in zebra-error and
    zebra-output.
[2] Remove zebra_loglevels from koha-conf (if present).
    Start/restart Zebra. Search to verify if Zebra runs.
    Stop Zebra and check zebra-error for a new line (killed by signal 15).
[3] Add fatal,warn in koha-conf:zebra_loglevels (do not include 'none').
    Start Zebra. Check zebra-output for a line "[log] zebra_start".
    Do a search and check zebra-output for lines with "[request]".
[4] Verify that zebra.log did not appear again.

Signed-off-by: Mark Tompsett <mtompset@hotmail.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 18 Jonathan Druart 2016-02-16 10:30:39 UTC
Created attachment 48105 [details] [review]
Bug 15714: Add zebra_loglevels to koha-conf.xml

This patch adds a commented zebra_loglevels line to the config file.
Behavior does not change, but an admin can easily edit the variable.

Test plan:
Nothing really changed. But if you are not so sure about that, apply the
change to /etc/koha/koha-conf-site.xml.in and run koha-create. Check
that the new koha-conf contains the line and is not messed up.

Signed-off-by: Mark Tompsett <mtompset@hotmail.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 19 David Cook 2016-02-16 23:22:02 UTC
Hmm... is koha-conf.xml really the most appropriate place to put <zebra_loglevels>, since it's only going to be able to affect Debian package installs?

I notice that it is in koha-conf.xml.in, so it should only appear in koha-conf.xml files generated by the Debian packages... so I guess that's OK... but it represents yet another deviation between the Debian package installs and all other Koha installs...
Comment 20 Marcel de Rooy 2016-02-17 12:18:25 UTC
(In reply to David Cook from comment #19)

Hi David. Thx for your interest in this patch.

> Hmm... is koha-conf.xml really the most appropriate place to put
> <zebra_loglevels>, since it's only going to be able to affect Debian package
> installs?

Hmm only Debian installs :) Your second remark is part of the answer btw.

> I notice that it is in koha-conf.xml.in, so it should only appear in
> koha-conf.xml files generated by the Debian packages... so I guess that's
> OK... but it represents yet another deviation between the Debian package
> installs and all other Koha installs...

This is an interesting topic but really outside the scope of this tiny change.
The percentage of Debian users of Koha will probably be quite substantial due to the tools developed for it. (We eventually switched from Fedora to Debian for such reasons.)
The various install modes (default, single, dev), Debian packaging, gitified, having possibly different directory structures etc., imo really beg for being simplified and brough together. But realizing that is hard. In the meantime the Debian percentage will keep growing?
Comment 21 David Cook 2016-02-18 05:52:36 UTC
(In reply to Marcel de Rooy from comment #20)
> This is an interesting topic but really outside the scope of this tiny
> change.
> The percentage of Debian users of Koha will probably be quite substantial
> due to the tools developed for it. (We eventually switched from Fedora to
> Debian for such reasons.)
> The various install modes (default, single, dev), Debian packaging,
> gitified, having possibly different directory structures etc., imo really
> beg for being simplified and brough together. But realizing that is hard. In
> the meantime the Debian percentage will keep growing?

I think you're right. I think it's a worthwhile change :).

I admit that I think about switching away from openSUSE to Debian for this reason as well, although we also run a few different versions of Koha on the same server, and that's something that the packages can't cope with I think.

I would love if the different installation streams could use the same tools, but it is hard, and the Debian percentage probably is growing (and with good reason). 

Thanks for taking the time to reply :).
Comment 22 Brendan Gallagher 2016-02-24 03:51:54 UTC
Pushed to Master - Should be in the may 2016 release.  Thanks much!
Comment 23 Marcel de Rooy 2017-01-30 08:49:25 UTC
Barton: We do not need to add zebra_loglevels to etc/koha-conf.xml here, since zebra_loglevels is only used in debian script koha-functions.sh, not used by other install types.