We have a Koha instance where records are not being indexed into Elasticsearch when they are first catalogued. They can be accessed if you go to the catalogue detail page directly, but cannot be searched. However if you edit the record and save, it is indexed successfully.
We've come up with a hacky fix for this because I can't think of any other reason indexing isn't working, but I am really happy to be advised about this.
Created attachment 146999 [details] [review] Bug 33019: Make sure ES records are indexed the first time This patch ensures records are indexed when they are created.
I have no idea if it's related yet, but there was also bug 33008 filed a little while ago. Could it be a similar issue?
Hi Aleisha, is the hack adding an item? if so it seems that it is related to https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33008 cheers
Hi all, Yes adding an item does cause the record to be indexed, but this Koha instance does not use items (they are a cataloguing agency). Also it had nothing to do with Zebra being started or stopped, I tried cataloguing with Zebra on and with Zebra off and it had no effect. Aleisha
Hi, I hope https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33243 helps, you may use it in a cronjob eg every 5 minutes rebuild_elasticsearch.pl -v -b --minutes 5 if you find in helpful and it works for you, please sign it off thx Mark
Created attachment 148444 [details] [review] Bug 33019: Make sure ES records are indexed the first time This patch ensures records are indexed when they are created. Previously, we were launching the indexer inside of of a transaction. This meant that the job was being enqueued, but not being found by the worker, becaue it was not yet in the DB This patch skips record indexing in the transaction, and moves the indexing command after To test: 1 - Make sure you are using ES, and the es indexer is running 2 - tail -f /var/log/koha/kohadev/*.log 3 - Create a new record 4 - Note error in es-indexer-output.log like: [2023/03/21 12:22:36] [WARN] No job found for id=157 main:: /kohadevbox/koha/misc/workers/es_indexer_daemon.pl (129) 5 - Apply patch 6 - Create another record 7 There should be no error 8 - Search for the record and confirm it can be found 9 - View the background jobs in admin, confirm the most recent job has completed
(In reply to Aleisha Amohia from comment #1) > We've come up with a hacky fix for this because I can't think of any other > reason indexing isn't working, but I am really happy to be advised about > this. I amended the patch, and added some comments, could you test Aleisha? It turns out the issue was that we were inside a transaction
Created attachment 148447 [details] [review] Bug 33019: Make sure ES records are indexed the first time This patch ensures records are indexed when they are created. Previously, we were launching the indexer inside of of a transaction. This meant that the job was being enqueued, but not being found by the worker, becaue it was not yet in the DB This patch skips record indexing in the transaction, and moves the indexing command after To test: 1 - Make sure you are using ES, and the es indexer is running 2 - tail -f /var/log/koha/kohadev/*.log 3 - Create a new record 4 - Note error in es-indexer-output.log like: [2023/03/21 12:22:36] [WARN] No job found for id=157 main:: /kohadevbox/koha/misc/workers/es_indexer_daemon.pl (129) 5 - Apply patch 6 - Create another record 7 There should be no error 8 - Search for the record and confirm it can be found 9 - View the background jobs in admin, confirm the most recent job has completed
Created attachment 148572 [details] [review] Bug 33019: Make sure ES records are indexed the first time This patch ensures records are indexed when they are created. Previously, we were launching the indexer inside of of a transaction. This meant that the job was being enqueued, but not being found by the worker, becaue it was not yet in the DB This patch skips record indexing in the transaction, and moves the indexing command after To test: 1 - Make sure you are using ES, and the es indexer is running 2 - tail -f /var/log/koha/kohadev/*.log 3 - Create a new record 4 - Note error in es-indexer-output.log like: [2023/03/21 12:22:36] [WARN] No job found for id=157 main:: /kohadevbox/koha/misc/workers/es_indexer_daemon.pl (129) 5 - Apply patch 6 - Create another record 7 There should be no error 8 - Search for the record and confirm it can be found 9 - View the background jobs in admin, confirm the most recent job has completed Signed-off-by: Aleisha Amohia <aleishaamohia@hotmail.com>
Created attachment 148591 [details] [review] Bug 33019: Make sure ES records are indexed the first time This patch ensures records are indexed when they are created. Previously, we were launching the indexer inside of of a transaction. This meant that the job was being enqueued, but not being found by the worker, becaue it was not yet in the DB This patch skips record indexing in the transaction, and moves the indexing command after To test: 1 - Make sure you are using ES, and the es indexer is running 2 - tail -f /var/log/koha/kohadev/*.log 3 - Create a new record 4 - Note error in es-indexer-output.log like: [2023/03/21 12:22:36] [WARN] No job found for id=157 main:: /kohadevbox/koha/misc/workers/es_indexer_daemon.pl (129) 5 - Apply patch 6 - Create another record 7 There should be no error 8 - Search for the record and confirm it can be found 9 - View the background jobs in admin, confirm the most recent job has completed Signed-off-by: Aleisha Amohia <aleishaamohia@hotmail.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Created attachment 148592 [details] [review] Bug 33019: Make sure ES records are indexed the first time This patch ensures records are indexed when they are created. Previously, we were launching the indexer inside of of a transaction. This meant that the job was being enqueued, but not being found by the worker, becaue it was not yet in the DB This patch skips record indexing in the transaction, and moves the indexing command after To test: 1 - Make sure you are using ES, and the es indexer is running 2 - tail -f /var/log/koha/kohadev/*.log 3 - Create a new record 4 - Note error in es-indexer-output.log like: [2023/03/21 12:22:36] [WARN] No job found for id=157 main:: /kohadevbox/koha/misc/workers/es_indexer_daemon.pl (129) 5 - Apply patch 6 - Create another record 7 There should be no error 8 - Search for the record and confirm it can be found 9 - View the background jobs in admin, confirm the most recent job has completed Signed-off-by: Aleisha Amohia <aleishaamohia@hotmail.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
I guess that's coming from bug 30822.
Pushed to master for 23.05. Nice work everyone, thanks!
Nice work everyone! Pushed to stable for 22.11.x
Missing dependencies for 22.05.x, no backport.