Bug 39605 - We should resubmit an elasticsearch index job that fails
Summary: We should resubmit an elasticsearch index job that fails
Status: Failed QA
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching - Elasticsearch (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal
Assignee: Nick Clemens (kidclamp)
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2025-04-10 18:19 UTC by Nick Clemens (kidclamp)
Modified: 2025-04-14 07:03 UTC (History)
3 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Trivial patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:


Attachments
Bug 39605: Resubmit failed ES background jobs (1.60 KB, patch)
2025-04-10 18:35 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Clemens (kidclamp) 2025-04-10 18:19:18 UTC
Currently, we try/catch our batched ES updates and mark all the jobs we batched as finished no matter the outcome:
es_indexer_daemon.pl:
            try {
                $biblio_indexer->update_index( \@bib_chunk );
            } catch {
                $logger->warn( sprintf "Update of elastic index failed with: %s", $_ );
            };
.
.
.
    # Finish
    $jobs->update({
        progress => 1,
        status => 'finished',
        ended_on => \'NOW()',
    });


In the catch we should create a new background job to resubmit the failed jobs
Comment 1 Nick Clemens (kidclamp) 2025-04-10 18:35:41 UTC
Created attachment 180800 [details] [review]
Bug 39605: Resubmit failed ES background jobs

This patch adds a resubmission of an indexing chunk that fails as a new background job

To test:
0 - Have KTD using ES
1 - Apply patch, restart all
2 - Update a record
3 - Check background jobs and confirm it worked
4 - Edit koha-conf and set ES port to 9201, restart_all
5 - Update a record
6 - Check background jobs
7 - The job should be failing and creating new jobs
8 - Edit koha-conf and fix the port, restart_all
9 - Confirm the newest job now succeeds
Comment 2 David Cook 2025-04-11 02:40:32 UTC
If there's a problem with the record, couldn't this lead to an infinite loop?
Comment 3 Magnus Enger 2025-04-14 07:03:58 UTC
(In reply to David Cook from comment #2)
> If there's a problem with the record, couldn't this lead to an infinite loop?

I thought the same thing. I think this needs an answer before we can consider signing off.