| Summary: | Make batch_size configurable for koha-es-indexer | ||
|---|---|---|---|
| Product: | Koha | Reporter: | Tomás Cohen Arazi (tcohen) <tomascohen> |
| Component: | Searching - Elasticsearch | Assignee: | Tomás Cohen Arazi (tcohen) <tomascohen> |
| Status: | Pushed to main --- | QA Contact: | Martin Renvoize (ashimema) <martin.renvoize> |
| Severity: | enhancement | ||
| Priority: | P5 - low | CC: | evelyn, kyle, martin.renvoize, mtj, tomascohen, tristin.stagg |
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | All | ||
| GIT URL: | Initiative type: | --- | |
| Sponsorship status: | --- | Comma delimited list of Sponsors: | |
| Crowdfunding goal: | 0 | Patch complexity: | Trivial patch |
| Documentation contact: | Documentation submission: | ||
| Text to go in the release notes: |
A new configuration entry has been introduced in `koha-conf.xml` to control how many Elasticsearch update tasks the indexer processes per iteration.
Previously, the indexer defaulted to a batch size of 10. While the `koha-es-indexer` command supported overriding this with the `--batch_size` parameter, there was no way to set a different default value.
This change is particularly beneficial for large sites, where the indexer may otherwise struggle to keep pace with the update queue during peak activity.
|
Version(s) released in: |
25.11.00
|
| Circulation function: | |||
| Attachments: |
Bug 40890: Make batch_size configurable for koha-es-indexer
Bug 40890: Make batch_size configurable for koha-es-indexer Bug 40890: Move fetching of batch size to within the instance loop Bug 40890: Make batch_size configurable for koha-es-indexer Bug 40890: Move fetching of batch size to within the instance loop |
||
|
Description
Tomás Cohen Arazi (tcohen)
2025-09-26 19:49:53 UTC
Created attachment 187014 [details] [review] Bug 40890: Make batch_size configurable for koha-es-indexer This patch adds a new configuration entry in `koha-conf.xml` for making it possible to set a different `batch_size` for `koha-es-indexer`. The current implementation has a default of *10* that can be overridden using the CLI parameter `--batch_size N`. Unfortunately, if you restart the server or the service without passing the parameter, it will be set back to 10. This patch solves that by adding a new config entry and making the script use it. The passed CLI parameter will keep overridding the configured value. To test: 1. Run: $ ktd --shell k$ koha-es-indexer --restart --batch_size 100 kohadev k$ ps -ef | grep elastic => SUCCESS: No errors. Notice '100' is passed to the called perl script 2. Run: k$ koha-es-indexer --restart kohadev k$ ps -ef | grep elastic => FAIL: Notice '10' is the new value 3. Apply this patch 4. Run: k$ reset_all 5. Repeat 1 and 2 => SUCCESS: '100' is kept. 6. Try standalone `--start` too 7. Sign off :-D Created attachment 188996 [details] [review] Bug 40890: Make batch_size configurable for koha-es-indexer This patch adds a new configuration entry in `koha-conf.xml` for making it possible to set a different `batch_size` for `koha-es-indexer`. The current implementation has a default of *10* that can be overridden using the CLI parameter `--batch_size N`. Unfortunately, if you restart the server or the service without passing the parameter, it will be set back to 10. This patch solves that by adding a new config entry and making the script use it. The passed CLI parameter will keep overridding the configured value. To test: 1. Run: $ ktd --shell k$ koha-es-indexer --restart --batch_size 100 kohadev k$ ps -ef | grep elastic => SUCCESS: No errors. Notice '100' is passed to the called perl script 2. Run: k$ koha-es-indexer --restart kohadev k$ ps -ef | grep elastic => FAIL: Notice '10' is the new value 3. Apply this patch 4. Run: k$ reset_all 5. Repeat 1 and 2 => SUCCESS: '100' is kept. 6. Try standalone `--start` too 7. Sign off :-D Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Created attachment 188997 [details] [review] Bug 40890: Move fetching of batch size to within the instance loop Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Created attachment 189143 [details] [review] Bug 40890: Make batch_size configurable for koha-es-indexer This patch adds a new configuration entry in `koha-conf.xml` for making it possible to set a different `batch_size` for `koha-es-indexer`. The current implementation has a default of *10* that can be overridden using the CLI parameter `--batch_size N`. Unfortunately, if you restart the server or the service without passing the parameter, it will be set back to 10. This patch solves that by adding a new config entry and making the script use it. The passed CLI parameter will keep overridding the configured value. To test: 1. Run: $ ktd --shell k$ koha-es-indexer --restart --batch_size 100 kohadev k$ ps -ef | grep elastic => SUCCESS: No errors. Notice '100' is passed to the called perl script 2. Run: k$ koha-es-indexer --restart kohadev k$ ps -ef | grep elastic => FAIL: Notice '10' is the new value 3. Apply this patch 4. Run: k$ reset_all 5. Repeat 1 and 2 => SUCCESS: '100' is kept. 6. Try standalone `--start` too 7. Sign off :-D Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk> Created attachment 189144 [details] [review] Bug 40890: Move fetching of batch size to within the instance loop Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk> Nice work everyone! Pushed to main for 25.11 Could this be backported to 25.05? |