Bug 36702 - Background Jobs: Add option to retry failed jobs with a max tries parameter
Summary: Background Jobs: Add option to retry failed jobs with a max tries parameter
Status: NEW
Alias: None
Product: Koha
Classification: Unclassified
Component: Architecture, internals, and plumbing (show other bugs)
Version: unspecified
Hardware: All All
: P5 - low enhancement
Assignee: Tomás Cohen Arazi (tcohen)
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-04-25 19:29 UTC by Lisette Scheer
Modified: 2025-04-30 04:25 UTC (History)
5 users (show)

See Also:
GIT URL:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Lisette Scheer 2024-04-25 19:29:14 UTC
Currently the background jobs just fail. It would be helpful if there was a path to add or set background jobs with a parameter to retry x times (probably with a y seconds interval between?) especially now that plugins can add background jobs.
Comment 1 Tomás Cohen Arazi (tcohen) 2025-04-29 14:15:23 UTC
Would it be acceptable to make this feature depend on manually installing a RabbitMQ plugin?

https://github.com/rabbitmq/rabbitmq-delayed-message-exchange/releases

I don't know how to solve 'the STOMP use case' otherwise.
Comment 2 David Cook 2025-04-30 02:57:28 UTC
As noted on that RabbitMQ plugin link, having a task scheduler could solve the problem for both broker methods. 

Upon failure, a task could be enqueued to run X seconds later. That task could be to restart the background job. It could reset the job status and then for RabbitMQ mode it could send a new message. The number of background job tries could be tracked in the job. 

We need a task scheduler at some point anyway.
Comment 3 David Cook 2025-04-30 02:58:49 UTC
That said... what's our proposed use case? What's the background job that would benefit from an automatic retry?
Comment 4 Tomás Cohen Arazi (tcohen) 2025-04-30 03:03:14 UTC
(In reply to David Cook from comment #3)
> That said... what's our proposed use case? What's the background job that
> would benefit from an automatic retry?

I have some real life use cases

- Plugin hook scheduling API calls to external API that could fail (they do fail, not often but do) and I want the job to be able to detect certain errors and schedule a retry.
- ES overwhelmed somehow, I would like the ES indexing job to be retried under certain circumstances.
Comment 5 David Cook 2025-04-30 04:25:30 UTC
(In reply to Tomás Cohen Arazi (tcohen) from comment #4)
> - Plugin hook scheduling API calls to external API that could fail (they do
> fail, not often but do) and I want the job to be able to detect certain
> errors and schedule a retry.

To me, it sounds like this would benefit from a task scheduler, and then the retry logic is just part of the plugin. 

> - ES overwhelmed somehow, I would like the ES indexing job to be retried
> under certain circumstances.

That could probably be useful. I suppose many things where there is inter-process communication it can be handy to be able to retry up to X times. So long as there's some coding to make sure there's no race conditions. ES indexing should be fine since it should just be passing an ID rather than any stale data.

--

Yeah overall I think a task scheduler solves this.