Summary: | Improve speed of koha-preferences CLI tool (by lazy-loading modules) | ||
---|---|---|---|
Product: | Koha | Reporter: | David Cook <dcook> |
Component: | Command-line Utilities | Assignee: | David Cook <dcook> |
Status: | RESOLVED FIXED | QA Contact: | Martin Renvoize (ashimema) <martin.renvoize> |
Severity: | enhancement | ||
Priority: | P5 - low | CC: | julian.maurice, lucas, martin.renvoize, robin |
Version: | Main | ||
Hardware: | All | ||
OS: | All | ||
Change sponsored?: | --- | Patch complexity: | --- |
Documentation contact: | Documentation submission: | ||
Text to go in the release notes: |
By lazy-loading modules needed for some functions of the koha-preferences CLI tool, other functions which do not need those modules now run much faster.
|
Version(s) released in: |
24.11.00
|
Circulation function: | |||
Bug Depends on: | 37657 | ||
Bug Blocks: | |||
Attachments: |
Bug 37682: Lazy-load modules for setters in koha-preferences tool
Bug 37682: Lazy-load modules for setters in koha-preferences tool Bug 37682: Lazy-load modules for setters in koha-preferences tool |
Description
David Cook
2024-08-20 04:21:56 UTC
Created attachment 170491 [details] [review] Bug 37682: Lazy-load modules for setters in koha-preferences tool This change lazy-loads modules for setters in the koha-preferences tool, so that getters are free to run super fast. Between BZ 37657 and BZ 37682 we effectively eliminate the overhead of running "get" or "dump" commands via the koha-preferences tool. Test plan: 1. time misc/admin/koha-preferences get SearchEngine 2. Note time is about .35 seconds 3. time misc/admin/koha-preferences dump 4. Note time is about .35 seconds 5. Create sysprefs.yml --- marcflavour: MARC21 viewMARC: 1 6. time misc/admin/koha-preferences load -i sysprefs.yml 7. Note time is about .35 seconds 8. time misc/admin/koha-preferences set SearchEngine Elasticsearch 9. Note time is about 1 seconds 10. Apply patch 11. Repeat the koha-preferences commands above 12. Note that the "dump" and "get" commands run in about .09-.1 seconds. The "load" and "set" commands still take the same amount of time as their behaviours haven't changed 13. misc/admin/koha-preferences set SearchEngine Elasticsearch1 14. koha-mysql kohadev 15. select * from action_logs where module = 'SYSTEMPREFERENCE' order by action_id desc limit 5; 16. Note that the action log showing Elasticsearch1 update says "interface" of "commandline" and "script" of "koha-preferences" Created attachment 170521 [details] [review] Bug 37682: Lazy-load modules for setters in koha-preferences tool This change lazy-loads modules for setters in the koha-preferences tool, so that getters are free to run super fast. Between BZ 37657 and BZ 37682 we effectively eliminate the overhead of running "get" or "dump" commands via the koha-preferences tool. Test plan: 1. time misc/admin/koha-preferences get SearchEngine 2. Note time is about .35 seconds 3. time misc/admin/koha-preferences dump 4. Note time is about .35 seconds 5. Create sysprefs.yml Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Created attachment 170522 [details] [review] Bug 37682: Lazy-load modules for setters in koha-preferences tool This change lazy-loads modules for setters in the koha-preferences tool, so that getters are free to run super fast. Between BZ 37657 and BZ 37682 we effectively eliminate the overhead of running "get" or "dump" commands via the koha-preferences tool. Test plan: 1. time misc/admin/koha-preferences get SearchEngine 2. Note time is about .35 seconds 3. time misc/admin/koha-preferences dump 4. Note time is about .35 seconds 5. Create sysprefs.yml --- marcflavour: MARC21 viewMARC: 1 6. time misc/admin/koha-preferences load -i sysprefs.yml 7. Note time is about .35 seconds 8. time misc/admin/koha-preferences set SearchEngine Elasticsearch 9. Note time is about 1 seconds 10. Apply patch 11. Repeat the koha-preferences commands above 12. Note that the "dump" and "get" commands run in about .09-.1 seconds. The "load" and "set" commands still take the same amount of time as their behaviours haven't changed 13. misc/admin/koha-preferences set SearchEngine Elasticsearch1 14. koha-mysql kohadev 15. select * from action_logs where module = 'SYSTEMPREFERENCE' order by action_id desc limit 5; 16. Note that the action log showing Elasticsearch1 update says "interface" of "commandline" and "script" of "koha-preferences" Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Another one I feel confident going straight for QA on.. all works as described and the speed boost is significant. Note that Koha::Script uses C4::Context, so C4::Context::import is executed twice (once with require Koha::Script, once with C4::Context->import) Koha::Logger::import is also "called" twice (it doesn't exist so it doesn't matter much IMO) And you might want to get rid of C4::Log which does not appear to be useful here. (In reply to Julian Maurice from comment #5) > Note that Koha::Script uses C4::Context, so C4::Context::import is executed > twice (once with require Koha::Script, once with C4::Context->import) > Koha::Logger::import is also "called" twice (it doesn't exist so it doesn't > matter much IMO) In my testing, "use Koha::Script" followed by "use C4::Context" calls C4::Context->import" twice as well, so same same. I haven't investigated Koha::Logger but better safe than sorry. Again, "use Koha::Logger" does the same thing. > And you might want to get rid of C4::Log which does not appear to be useful > here. I'm just doing a like for like so far. Happy for someone to do a follow up. Note: I'm running this already in production, and it's wonderful. (In reply to David Cook from comment #7) > Note: I'm running this already in production, and it's wonderful. Just to be clear: I didn't mean to block the patch from being pushed and just wanted to point out what could be improved. (In reply to David Cook from comment #6) > In my testing, "use Koha::Script" followed by "use C4::Context" > calls C4::Context->import" twice as well, so same same. IIUC the 2nd call is useless but it's exactly like before the patch, so no problem here. (In reply to David Cook from comment #7) > Note: I'm running this already in production, and it's wonderful. Could you explain a bit on how you use it? Just interested in use cases. :) Pushed for 24.11! Well done everyone, thank you! (In reply to Katrin Fischer from comment #9) > (In reply to David Cook from comment #7) > > Note: I'm running this already in production, and it's wonderful. > > Could you explain a bit on how you use it? Just interested in use cases. :) It's used by "koha-list --elasticsearch", which is invoked by the "koha-common" service. During a "apt-get install koha-common", it actually gets run twice through the process, so it can save a lot of time for deployments. (In reply to David Cook from comment #11) > (In reply to Katrin Fischer from comment #9) > > (In reply to David Cook from comment #7) > > > Note: I'm running this already in production, and it's wonderful. > > > > Could you explain a bit on how you use it? Just interested in use cases. :) > > It's used by "koha-list --elasticsearch", which is invoked by the > "koha-common" service. > > During a "apt-get install koha-common", it actually gets run twice through > the process, so it can save a lot of time for deployments. I understand, thanks! |