Bug 30260

Summary: Script which checks index should also check if the zebraqueue has uncommitted records
Product: Koha Reporter: Aleisha Amohia <aleisha>
Component: Architecture, internals, and plumbingAssignee: Aleisha Amohia <aleisha>
Status: RESOLVED DUPLICATE QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: david, martin.renvoize, tomascohen
Version: Main   
Hardware: All   
OS: All   
Change sponsored?: Sponsored Patch complexity: Small patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:
Attachments: Bug 30260: Add is_indexer_working check
Bug 30260: [GLOBAL REPO] Implement is_indexer_working check
Bug 30260: Add is_indexer_working check

Description Aleisha Amohia 2022-03-10 02:11:29 UTC
currently our monitoring checks if the indexer is running, and sends an alert if it it isn't.

however we've found that sometimes the indexer looks like it is running but the process may not have started fully, and the indexer isn't working. new records and modified records don't show in search results. when we view the zebraqueue in the database, there are rows that have not been committed, i.e. `zebraqueue.done` == 0.

checkindex.sh should check if there are rows in the zebraqueue that have not been committed.
Comment 1 Aleisha Amohia 2022-03-10 02:48:31 UTC
Created attachment 131532 [details] [review]
Bug 30260: Add is_indexer_working check

Function checks if the zebraqueue has rows that are not 'done'

Will depend on a change in checkindex.sh which implements the check: https://git.koha-community.org/Koha-community/global/src/branch/master/koha-zebra/checkindex.sh

To test:

1. disable the indexer

sudo koha-indexer --stop INSTANCENAME

2. catalogue 5 new records
3. confirm they don't show in any search results
4. in your terminal, go to the shell

sudo koha-shell INSTANCENAME

5. run the new script. run with -v for verbose or --rows X to specify
how many uncommitted records should be checked for. 5 is the default

perl misc/maintenance/check_zebraqueue.pl

6. confirm the script returns that the indexer is not working.

The function in koha-functions.sh is required for the pending change to
checkindex.sh

Sponsored-by: Catalyst IT
Comment 2 Aleisha Amohia 2022-03-10 22:19:08 UTC
Unfortunately I can't seem to push to the repo, can someone please help? I can attach the patch here but it won't apply to a koha clone.

Am trying to push to this repo: https://git.koha-community.org/Koha-community/global/
Comment 3 Aleisha Amohia 2022-03-10 22:24:46 UTC
Created attachment 131582 [details] [review]
Bug 30260: [GLOBAL REPO] Implement is_indexer_working check
Comment 4 Katrin Fischer 2022-07-18 21:28:06 UTC
Tomas or Martin, can you help with pushing to https://git.koha-community.org/Koha-community/global/?
Comment 5 Tomás Cohen Arazi (tcohen) 2022-07-18 22:27:52 UTC
I had no idea that repo existed.
Comment 6 Aleisha Amohia 2022-10-03 04:04:29 UTC
How should I get that patch into the global repo?
Comment 7 David Nind 2024-01-02 10:35:24 UTC
Created attachment 160395 [details] [review]
Bug 30260: Add is_indexer_working check

Function checks if the zebraqueue has rows that are not 'done'

Will depend on a change in checkindex.sh which implements the check: https://git.koha-community.org/Koha-community/global/src/branch/master/koha-zebra/checkindex.sh

To test:

1. disable the indexer

sudo koha-indexer --stop INSTANCENAME

2. catalogue 5 new records
3. confirm they don't show in any search results
4. in your terminal, go to the shell

sudo koha-shell INSTANCENAME

5. run the new script. run with -v for verbose or --rows X to specify
how many uncommitted records should be checked for. 5 is the default

perl misc/maintenance/check_zebraqueue.pl

6. confirm the script returns that the indexer is not working.

The function in koha-functions.sh is required for the pending change to
checkindex.sh

Sponsored-by: Catalyst IT
Signed-off-by: David Nind <david@davidnind.com>
Comment 8 David Nind 2024-01-02 10:38:17 UTC
Patch still applies and it worked for me: I had to add 6 new records for the default of 5 to work.

Maybe check with Jonathan about the repo - probably something to do with access when we changed to Gitea. Maybe the repository should be added to gitlab.com/koha-community
Comment 9 Aleisha Amohia 2024-01-04 22:24:10 UTC
Comment on attachment 131582 [details] [review]
Bug 30260: [GLOBAL REPO] Implement is_indexer_working check

This patch has now been pushed to the global repo, so the other patch can be pushed to Koha.

https://git.koha-community.org/Koha-community/global/commit/72308768457017c3ff48543c9a52295a064a7253
Comment 10 Aleisha Amohia 2024-04-09 03:07:34 UTC
I don't think we need this fix anymore, with a combination of the following fixes instead:

* Bug 36548 (the bug solved here was the believed cause for this fix)
* https://git.koha-community.org/Koha-community/global/commit/704060f4ff5497b545b9d807e2ea57c5cc7a85b9

*** This bug has been marked as a duplicate of bug 36548 ***