Bug 20745 - indexing/searching not active at end of installation
Summary: indexing/searching not active at end of installation
Status: RESOLVED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Installation and upgrade (web-based installer) (show other bugs)
Version: master
Hardware: PC Linux
: P1 - high major with 10 votes (vote)
Assignee: Tomás Cohen Arazi
QA Contact: Testopia
URL:
Whiteboard:
Keywords:
Depends on: 20234
Blocks:
  Show dependency treegraph
 
Reported: 2018-05-09 13:52 UTC by Giuseppe Ciaccio
Modified: 2018-07-09 10:35 UTC (History)
7 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Trivial patch
Bot Control: ---
When did the bot last check this:
Who signed the patch off:
Text to go in the release notes:


Attachments
output of "ps ax" before and after koha-common service restart; note the initial absence of koha-indexer (libraryname = ferrer) (2.13 KB, text/plain)
2018-05-10 11:00 UTC, Giuseppe Ciaccio
Details
Bug 20745: koha-zebra doesn't return the correct error codes (2.87 KB, patch)
2018-05-10 15:44 UTC, Tomás Cohen Arazi
Details | Diff | Splinter Review
Bug 20745: koha-zebra doesn't return the correct error codes (2.92 KB, patch)
2018-05-10 16:05 UTC, M. Tompsett
Details | Diff | Splinter Review
Bug 20745: koha-zebra doesn't return the correct error codes (2.97 KB, patch)
2018-05-10 16:35 UTC, Katrin Fischer
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Giuseppe Ciaccio 2018-05-09 13:52:25 UTC
After installing Koha on Debian (instructions at https://wiki.koha-community.org/wiki/Koha_on_Debian) with marc21 and english language, and completing the installation wizard, you are finally redirected to admin login and then you can perform your first Koha admin operation.  Suppose you insert a biblio record with related item -- a typical first step for the beginner.  At this point the item is indeed there (e.g. it appears in inventory) but is not searchable: the search page for common users always returns "not found".  And then you cry :)

But if, at this point, you open a shell on the server and issue the commands

sudo service koha-common stop
sudo service koha-common start

then everything starts working correctly, and the inserted item appears in the search results.  Subsequently entered records are immediately searchable as well.

It looks like the install wizard omits a final step consisting of activating the indexing service, or something like that.  Or, otherwise, the install instructions are omitting the above commands for restarting service before first operation.
Comment 1 Giuseppe Ciaccio 2018-05-09 14:35:27 UTC
Important remark:

I've read somewhere that the Koha indexing may incur delays, or happens periodically, and therefore a recently inserted record does not appear immediately in the search but will do after some minutes.  This is not the case here: after restarting the koha-common services, I see that each new record appears immediately in the search results ("immediately" means "less than 5 seconds after insert").
Comment 2 Jonathan Druart 2018-05-09 17:48:02 UTC
Hum, I do not know what your missing here but it should just work.

It's not related to the installation process, the koha-create command should start a koha-indexer process to take care of the indexation.

Maybe it did not start correctly?
Comment 3 Giuseppe Ciaccio 2018-05-09 19:17:13 UTC
As a further detail: in my attempts I always start from a fresh Debian 9.4 default install, to be sure nothing has remained from past attempts (this means: all partitions are formatted and then the OS is installed, and finally I install Koha following very strictly the instructions at https://wiki.koha-community.org/wiki/Debian). As of now I have tried with marc21 flavour and english, and marc21 and it-IT, with same results. This evening I will try with unimarc flavour.
Comment 4 Giuseppe Ciaccio 2018-05-10 10:58:37 UTC
I've done a test with unimarc flavour and english or it-IT as the installation language, nothing changes.  The inserted records become searchable only after "service koha-common stop && service koha-common start".

However, after completing the install/config wizard, I looked at the output of "ps ax" before and after service restart, and noted that, after restart, a new daemon process appeared, called <libraryname>-koha-indexer, which was missing before.  I will send a logfile to show this.

IMHO the wizard, or the initial koha-create command, fails launching the koha-indexer, and that's why the indexing/searching provides no results until the service is manually restarted.
Comment 5 Giuseppe Ciaccio 2018-05-10 11:00:01 UTC
Created attachment 75217 [details]
output of "ps ax" before and after koha-common service restart; note the initial absence of koha-indexer (libraryname = ferrer)
Comment 6 Jonathan Druart 2018-05-10 13:11:03 UTC
% sudo koha-create --create-db xxx
Koha instance is empty, no staff user created.
% ps aux | grep xxx-koha-zebra
xxx-koha  3534  0.1  0.0  18608   188 ?        S    10:09   0:00 daemon --name=xxx-koha-zebra --pidfiles=/var/run/koha/xxx/ --errlog=/var/log/koha/xxx/zebra-error.log --output=/var/log/koha/xxx/zebra-output.log --verbose=1 --respawn --delay=30 --user=xxx-koha.xxx-koha -- /usr/bin/zebrasrv -v none,fatal,warn -f /etc/koha/sites/xxx/koha-conf.xml

Looks ok for me.
Comment 7 Tomás Cohen Arazi 2018-05-10 13:12:29 UTC
(In reply to Jonathan Druart from comment #6)
> % sudo koha-create --create-db xxx
> Koha instance is empty, no staff user created.
> % ps aux | grep xxx-koha-zebra
> xxx-koha  3534  0.1  0.0  18608   188 ?        S    10:09   0:00 daemon
> --name=xxx-koha-zebra --pidfiles=/var/run/koha/xxx/
> --errlog=/var/log/koha/xxx/zebra-error.log
> --output=/var/log/koha/xxx/zebra-output.log --verbose=1 --respawn --delay=30
> --user=xxx-koha.xxx-koha -- /usr/bin/zebrasrv -v none,fatal,warn -f
> /etc/koha/sites/xxx/koha-conf.xml
> 
> Looks ok for me.

what about the indexer?
Comment 8 Jonathan Druart 2018-05-10 13:15:22 UTC
heh, it's missing :)
Comment 9 Tomás Cohen Arazi 2018-05-10 13:26:36 UTC
(In reply to Jonathan Druart from comment #8)
> heh, it's missing :)

--- a/misc/migration_tools/rebuild_zebra.pl
+++ b/misc/migration_tools/rebuild_zebra.pl
@@ -242,9 +242,14 @@ if ($daemon_mode) {
         if (_flock($LockFH, LOCK_EX|LOCK_NB)) {
             eval {
                 $dbh = C4::Context->dbh;
-                if( zebraqueue_not_empty() ) {
-                    Koha::Caches->flush_L1_caches() if $is_memcached;
-                    do_one_pass();
+                if ( C4::Context->preference('Version') ) {
+                    if( zebraqueue_not_empty() ) {
+                        Koha::Caches->flush_L1_caches() if $is_memcached;
+                        do_one_pass();
+                    }
+                }
+                else {
+                    warn "Warning: Koha is not installed.";
                 }
Comment 10 Jonathan Druart 2018-05-10 13:45:07 UTC
To my understanding the problem is in koha-zebra.

koha-create calls: koha-zebra --start "$name"

And then exit 1

it's because the function start_zebra (from koha-zebra) returns an error with:
   [ "$verbose" != "no" ] && \
       log_end_msg 0
Comment 11 Jonathan Druart 2018-05-10 13:56:38 UTC
Certainly caused by bug 20234.
Comment 12 Tomás Cohen Arazi 2018-05-10 15:44:24 UTC
Created attachment 75227 [details] [review]
Bug 20745: koha-zebra doesn't return the correct error codes

This patch makes koha-zebra return 0 (or 1) even when verbose mode is
off. This made koha-zebra --start always return a failure even when the
daemon was correctly launched.

To test:
- Have Zebra running for kohadev instance on your KohaDevBox
- Run:
  $ sudo koha-zebra --stop kohadev && echo yay || echo doh
=> FAIL: 'doh' is printed
- Run:
  $ sudo koha-zebra --start kohadev e&& echo yay || echo doh
=> FAIL: 'doh' is printed as well
- Apply this patch
- Run:
  $ sudo perl misc4dev/cp_debian_files.pl
  $ sudo koha-zebra --stop kohadev && echo yay || echo doh
=> SUCCESS: 'yay' is printed
- Run:
  $ sudo koha-zebra --start kohadev e&& echo yay || echo doh
=> SUCCESS: 'yay' is printed
- Sign off :-D

Sponsored-by: ByWater Solutions
Comment 13 Tomás Cohen Arazi 2018-05-10 15:45:14 UTC
Added the 17.11 RMaint because the original patches got backported.
Comment 14 M. Tompsett 2018-05-10 16:05:04 UTC
Created attachment 75231 [details] [review]
Bug 20745: koha-zebra doesn't return the correct error codes

This patch makes koha-zebra return 0 (or 1) even when verbose mode is
off. This made koha-zebra --start always return a failure even when the
daemon was correctly launched.

To test:
- Have Zebra running for kohadev instance on your KohaDevBox
- Run:
  $ sudo koha-zebra --stop kohadev && echo yay || echo doh
=> FAIL: 'doh' is printed
- Run:
  $ sudo koha-zebra --start kohadev e&& echo yay || echo doh
=> FAIL: 'doh' is printed as well
- Apply this patch
- Run:
  $ sudo perl misc4dev/cp_debian_files.pl
  $ sudo koha-zebra --stop kohadev && echo yay || echo doh
=> SUCCESS: 'yay' is printed
- Run:
  $ sudo koha-zebra --start kohadev e&& echo yay || echo doh
=> SUCCESS: 'yay' is printed
- Sign off :-D

Sponsored-by: ByWater Solutions

Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Comment 15 Katrin Fischer 2018-05-10 16:35:13 UTC
Created attachment 75233 [details] [review]
Bug 20745: koha-zebra doesn't return the correct error codes

This patch makes koha-zebra return 0 (or 1) even when verbose mode is
off. This made koha-zebra --start always return a failure even when the
daemon was correctly launched.

To test:
- Have Zebra running for kohadev instance on your KohaDevBox
- Run:
  $ sudo koha-zebra --stop kohadev && echo yay || echo doh
=> FAIL: 'doh' is printed
- Run:
  $ sudo koha-zebra --start kohadev e&& echo yay || echo doh
=> FAIL: 'doh' is printed as well
- Apply this patch
- Run:
  $ sudo perl misc4dev/cp_debian_files.pl
  $ sudo koha-zebra --stop kohadev && echo yay || echo doh
=> SUCCESS: 'yay' is printed
- Run:
  $ sudo koha-zebra --start kohadev e&& echo yay || echo doh
=> SUCCESS: 'yay' is printed
- Sign off :-D

Sponsored-by: ByWater Solutions

Signed-off-by: Mark Tompsett <mtompset@hotmail.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 16 Jonathan Druart 2018-05-11 14:13:23 UTC
Pushed to master for 18.05, thanks to everybody involved!
Comment 17 Mason James 2018-07-09 06:30:18 UTC
(In reply to Tomás Cohen Arazi from comment #13)
> Added the 17.11 RMaint because the original patches got backported.

Frido, just confirming this bug exists in 17.11.07

the patch fixes the bug :)
Comment 18 Fridolin SOMERS 2018-07-09 09:51:23 UTC
(In reply to Mason James from comment #17)
> (In reply to Tomás Cohen Arazi from comment #13)
> > Added the 17.11 RMaint because the original patches got backported.
> 
> Frido, just confirming this bug exists in 17.11.07
> 
> the patch fixes the bug :)

Ohhh thanks for this, I will backport
Comment 19 Fridolin SOMERS 2018-07-09 10:34:44 UTC
Pushed to 17.11.x for 17.11.08
Comment 20 Fridolin SOMERS 2018-07-09 10:35:31 UTC
Depends on Bug 20234 not in 17.05.x