| Summary: | Make batch_size configurable for koha-es-indexer | ||
|---|---|---|---|
| Product: | Koha | Reporter: | Tomás Cohen Arazi (tcohen) <tomascohen> |
| Component: | Searching - Elasticsearch | Assignee: | Tomás Cohen Arazi (tcohen) <tomascohen> |
| Status: | Pushed to main --- | QA Contact: | Martin Renvoize (ashimema) <martin.renvoize> |
| Severity: | enhancement | ||
| Priority: | P5 - low | CC: | evelyn, kyle, martin.renvoize, mtj, tomascohen, tristin.stagg |
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | All | ||
| GIT URL: | Initiative type: | --- | |
| Sponsorship status: | --- | Comma delimited list of Sponsors: | |
| Crowdfunding goal: | 0 | Patch complexity: | Trivial patch |
| Documentation contact: | Documentation submission: | ||
| Text to go in the release notes: |
A new configuration entry has been introduced in `koha-conf.xml` to control how many Elasticsearch update tasks the indexer processes per iteration.
Previously, the indexer defaulted to a batch size of 10. While the `koha-es-indexer` command supported overriding this with the `--batch_size` parameter, there was no way to set a different default value.
This change is particularly beneficial for large sites, where the indexer may otherwise struggle to keep pace with the update queue during peak activity.
|
Version(s) released in: |
25.11.00
|
| Circulation function: | |||
| Attachments: |
Bug 40890: Make batch_size configurable for koha-es-indexer
Bug 40890: Make batch_size configurable for koha-es-indexer Bug 40890: Move fetching of batch size to within the instance loop Bug 40890: Make batch_size configurable for koha-es-indexer Bug 40890: Move fetching of batch size to within the instance loop |
||
|
Description
Tomás Cohen Arazi (tcohen)
2025-09-26 19:49:53 UTC
Created attachment 187014 [details] [review] Bug 40890: Make batch_size configurable for koha-es-indexer This patch adds a new configuration entry in `koha-conf.xml` for making it possible to set a different `batch_size` for `koha-es-indexer`. The current implementation has a default of *10* that can be overridden using the CLI parameter `--batch_size N`. Unfortunately, if you restart the server or the service without passing the parameter, it will be set back to 10. This patch solves that by adding a new config entry and making the script use it. The passed CLI parameter will keep overridding the configured value. To test: 1. Run: $ ktd --shell k$ koha-es-indexer --restart --batch_size 100 kohadev k$ ps -ef | grep elastic => SUCCESS: No errors. Notice '100' is passed to the called perl script 2. Run: k$ koha-es-indexer --restart kohadev k$ ps -ef | grep elastic => FAIL: Notice '10' is the new value 3. Apply this patch 4. Run: k$ reset_all 5. Repeat 1 and 2 => SUCCESS: '100' is kept. 6. Try standalone `--start` too 7. Sign off :-D Created attachment 188996 [details] [review] Bug 40890: Make batch_size configurable for koha-es-indexer This patch adds a new configuration entry in `koha-conf.xml` for making it possible to set a different `batch_size` for `koha-es-indexer`. The current implementation has a default of *10* that can be overridden using the CLI parameter `--batch_size N`. Unfortunately, if you restart the server or the service without passing the parameter, it will be set back to 10. This patch solves that by adding a new config entry and making the script use it. The passed CLI parameter will keep overridding the configured value. To test: 1. Run: $ ktd --shell k$ koha-es-indexer --restart --batch_size 100 kohadev k$ ps -ef | grep elastic => SUCCESS: No errors. Notice '100' is passed to the called perl script 2. Run: k$ koha-es-indexer --restart kohadev k$ ps -ef | grep elastic => FAIL: Notice '10' is the new value 3. Apply this patch 4. Run: k$ reset_all 5. Repeat 1 and 2 => SUCCESS: '100' is kept. 6. Try standalone `--start` too 7. Sign off :-D Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Created attachment 188997 [details] [review] Bug 40890: Move fetching of batch size to within the instance loop Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Created attachment 189143 [details] [review] Bug 40890: Make batch_size configurable for koha-es-indexer This patch adds a new configuration entry in `koha-conf.xml` for making it possible to set a different `batch_size` for `koha-es-indexer`. The current implementation has a default of *10* that can be overridden using the CLI parameter `--batch_size N`. Unfortunately, if you restart the server or the service without passing the parameter, it will be set back to 10. This patch solves that by adding a new config entry and making the script use it. The passed CLI parameter will keep overridding the configured value. To test: 1. Run: $ ktd --shell k$ koha-es-indexer --restart --batch_size 100 kohadev k$ ps -ef | grep elastic => SUCCESS: No errors. Notice '100' is passed to the called perl script 2. Run: k$ koha-es-indexer --restart kohadev k$ ps -ef | grep elastic => FAIL: Notice '10' is the new value 3. Apply this patch 4. Run: k$ reset_all 5. Repeat 1 and 2 => SUCCESS: '100' is kept. 6. Try standalone `--start` too 7. Sign off :-D Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk> Created attachment 189144 [details] [review] Bug 40890: Move fetching of batch size to within the instance loop Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk> Nice work everyone! Pushed to main for 25.11 Could this be backported to 25.05? I'd hold off on that.. right now this has taken down some servers where we backported it and I'm investigating OK.. it was a bad backport to 24.11.x for me.. feel free to go ahead |