Bug 8519 - Make koha-index-daemon run on startup
Summary: Make koha-index-daemon run on startup
Alias: None
Product: Koha
Classification: Unclassified
Component: Command-line Utilities (show other bugs)
Version: master
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Tomás Cohen Arazi
QA Contact:
Depends on:
Blocks: 23241
  Show dependency treegraph
Reported: 2012-07-30 12:36 UTC by Tomás Cohen Arazi
Modified: 2019-11-20 15:16 UTC (History)
6 users (show)

See Also:
Change sponsored?: Sponsored
Patch complexity: Small patch
Text to go in the release notes:
Version(s) released in:

Bug 8519 - Make koha-index-daemon run on startup (21.00 KB, patch)
2012-07-30 13:36 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
Bug 8519 - Make koha-index-daemon run on startup (21.70 KB, patch)
2012-07-30 14:25 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
Bug 8519 - Make koha-index-daemon run on startup (25.60 KB, patch)
2012-08-11 14:36 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
SIGNED-OFF-Bug-8519-Make-koha-index-daemon-run-on-st (25.70 KB, patch)
2012-08-11 19:05 UTC, wajasu
Details | Diff | Splinter Review
Bug 8519 - Remove zebraqueue_start.pl script (cleanup followup, not important for the bug) (8.10 KB, patch)
2012-08-11 23:00 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
0001-SIGNED-OFF-Bug-8519-Remove-zebraqueue_start.pl-scrip (8.16 KB, patch)
2012-08-21 18:16 UTC, wajasu
Details | Diff | Splinter Review
Bug 8519 - Make koha-index-daemon run on startup (6.78 KB, patch)
2012-10-31 19:19 UTC, Jared Camins-Esakov
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Tomás Cohen Arazi 2012-07-30 12:36:04 UTC

Comment 1 Tomás Cohen Arazi 2012-07-30 13:36:57 UTC Comment hidden (obsolete)
Comment 2 Tomás Cohen Arazi 2012-07-30 14:25:08 UTC Comment hidden (obsolete)
Comment 3 Tomás Cohen Arazi 2012-08-10 15:07:25 UTC
To test this patch, install Koha, create a symlink for this script:

On Ubuntu:

$ ln -s /usr/share/koha/bin/koha-zebra-indexdaemon-ctl.sh /etc/init.d/koha-zebra-indexdaemon
$ update-rc.d koha-zebra-indexdaemon defaults
$ service koha-zebra-indexdaemon start

- Check it started without errors in the logs (/var/log/koha/koha-indexdaemon*)
- Do some cataloguing (biblios, authorities) and see if they are properly indexed by zebra.
Comment 4 wajasu 2012-08-11 05:02:06 UTC
I attempted to test, but it had a few of problems, with my "standard" fresh install.

1) make install failed, complaining that:
 "Can't open blib/SCRIPT_DIR/koha-zebraqueue-ctl.sh for read at rewrite-config.PL line 184".
It seems the Makefile.PL processes it at line 525, but that was removed in the patch.

It probably needs to be changed refer to the koha-zebra-indexdaemon-ctl.sh to replace the rewrite_config fields.

2) I changed it in my test environment, and "make install" ran, and I symlinked/ran the script but got :
20120810 15:39:25 koha-zebra-indexdaemon-ctl-koha: client (pid 11935) exited with 1 status
20120810 15:39:25 koha-zebra-indexdaemon-ctl-koha: client (pid 11940) exited with 1 status
20120810 15:39:25 koha-zebra-indexdaemon-ctl-koha: terminating too quickly, waiting 30 seconds

koha-index-daemon is not anywhere to be found in the patch.

3) jcamins on IRC suggested that Koha:Contrib:Tamil contained the missing script, so I used cpan in my test vm. That included a lot of stuff.  Maybe we only require the Koha:Contrib:Tamil:Indexer  and Koha:Contrib:Tamil:IndexerDaemon components.

After that I got the koha-zebra-indexdaemon script running. But,
it was failing which is the default directory.  So I added to
misc/bin/koha-zebra-indexdaemon-ctl.sh as part of the INDEXDAEMONOPTS
--directory /var/tmp/$DBNAME-zebraqueue-index
which allowed the tamil contribs indexdaemon to create it.
I checked the zebraqueue tables and after the 30 second timeout, it
processed another z3950 cataloging import.


4) Other things to notice:
find kohaclonedir -name "*.*" -exec grep -Hn "zebraqueue" {} \;
to find other places that mentioned zebra queue related things, and saw a
This is probably obsolete as well, and should be removed.

The INSTALL texts need to be updated, and one refers you to the wiki
for configuration.  So that need to be addressed. Possibly as a followup patch.

I will let the daemon run over the weekend to see if it behaves. But maybe we want someone to write a load "test". 

I'm going to bring things up next week to see if anything I mentioned needs to be addressed from the patch writer and community.
Comment 5 Tomás Cohen Arazi 2012-08-11 14:36:46 UTC Comment hidden (obsolete)
Comment 6 wajasu 2012-08-11 19:05:14 UTC Comment hidden (obsolete)
Comment 7 Tomás Cohen Arazi 2012-08-11 23:00:24 UTC Comment hidden (obsolete)
Comment 8 Chris Cormack 2012-08-11 23:22:14 UTC
Yes it runs a cron job that runs rebuild zebra for each instance. So this shouldn't effect it at all its only removing the old deprecated daemon and adding a new optional daemon. The cron job method shouldn't be touched at all,  ideally an option to run the daemon with the packages would be good too. But it's mire important to not break existing functionality.
Comment 9 wajasu 2012-08-21 18:16:10 UTC Comment hidden (obsolete)
Comment 10 wajasu 2012-08-21 18:45:46 UTC
Observation: after the "Make-koha-index-daemon-run" and "Remove-zerbaqueue_start" were applied, I still found reference to zebraqueue_start.

From the kohaclone git directory:
find . -name "*.*" -exec grep -Hn "zebraqueue_start" {} \;
results in:
./C4/Biblio.pm:2660:# the table is emptied by misc/cronjobs/zebraqueue_start.pl script
./C4/Biblio.pm:2714:    # the table is emptied by misc/cronjobs/zebraqueue_start.pl script
./C4/AuthoritiesMarc.pm:835:      # indexing processes like zebraqueue_start.pl

so this must be removed/changed to prevent from confusing future development.

Comment 11 Tomás Cohen Arazi 2012-08-30 13:56:53 UTC
I'd like to tell that this patch would also be usefull in the case of using another daemon script (the one for SolR for example). This is only a init script.
Comment 12 Mason James 2012-09-16 13:44:03 UTC
ive added a new bug '8774' as a blocker for this
Comment 13 Tomás Cohen Arazi 2012-10-22 15:08:33 UTC
I wonder what with happen with this bug now we have packages for Fredericd's daemon
Comment 14 Jared Camins-Esakov 2012-10-31 19:19:00 UTC
Created attachment 13156 [details] [review]
Bug 8519 - Make koha-index-daemon run on startup

[This patch was split out from tcohen's excellent patches for bug 8519
--jcamins 2012/10/31]

Added a new koha-index-daemon-ctl.sh script that uses the
'koha-index-daemon' provided by Koha::Contrib::Tamil (install via CPAN)
to process the indexing queue ('zebraqueue' for now, 'indexqueue' in the
future I guess :-P).

This script could be easily modified in the future for using another
index queue processing script (a more abstract indexing script for both Solr
and Zebra, etc). I removed all zebra-ish stuff from it.

Documentation patches will be provided as followups, once Mark Tompsett's
docs are pushed to avoid continous rebasing.

Tested to work on an up-to-date Ubuntu 12.04.

Sponsored-by: Universidad Nacional de Córdoba

Signed-off-by: wajasu <matted-34813@mypacks.net>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Comment 15 Marcel de Rooy 2012-12-06 14:27:22 UTC
Dry QA comment:
Cannot test this patch, since it is dependent on distro and koha-index-daemon. (This will be the case for more members of QA team probably.) 
Things as /lib/lsb/init-functions and daemon are not always present.
I tested make test and make install. Make test did not finish properly, but that seems not to be related to this patch. Make install installed the new script in the kohadev bin directory.
As far as I know, the t/db_dependent/lib/KohaTest.pm is a deprecated test unit.
But I do not see any harm in pushing this patch. It will not break anything if you do not use it. It needs more documentation but that was mentioned already.

Setting to Passed QA for consideration by RM.
Comment 16 Jared Camins-Esakov 2012-12-08 14:45:59 UTC
This patch has been pushed to master.
Comment 17 Tomás Cohen Arazi 2013-05-21 22:51:55 UTC
This patch shipped with 3.12.0.