Any script using C4::Context is bound to have a huge memory footprint because of the dependencies. And even script that would do without C4::Context (somehow) are bound for the same as soon as they include one Koha:: module, because those will use C4::Context. A graph from v23.05 will summarize it better than any explanation: BackgroundJobs | v BackgroundJob (Net::Stomp : 5M) | | | v | Koha::Object (Mojo::JSON : 16M) v C4::Context (DateTime::TimeZone : 7M) | | (ZOOM : 8M) v | Koha::Config::Syspref | | v | C4::Log (calling back on C4::Context) | | v | Koha::ActionLogs | | v | Koha::ActionLog (calling back on C4::Context) | | v | Koha::Object | | v Koha::Caches (calling back on C4::Context)
This is a branch from Bug 28410 and Bug 28411, for earlier exchanges.
Created attachment 158361 [details] [review] Bug 35250: Use lazy loading to reduce memory footprint of C4/Context.pm This simply removes some 'use' declarations at the top of the file to instead use 'require' later on. The effect can be validated with a simple one-liner (provided KOHA_CONF and PERL5LIB being set) perl -e "use C4::Context; while(1){sleep(1);}" And monitoring the memoring usage in tools like 'top'
Created attachment 158362 [details] [review] Bug 35250: remove usages of Koha::Config::Syspref(s) from C4::Context Use straigth SQL queries to access systempreferences instead of relying on the ORM, thus removing all direct and indirect dependencies to Koha::Object To remove all dependencies between C4::Context and the db object model, we move the calls to set_preference and delete_prefence directly to the class object Koha::Config::SysPrefs C4::Context::preference remains but uses straight SQL C4::Context::set_preference moves to Koha::Config::SysPrefs::set_preference C4::Context::delete_preference moves to Koha::Config::SysPrefs::delete_preference
Created attachment 158363 [details] [review] Bug 35250: modify scripts calling C4::Context::set_preference All the scripts using the call C4::Context->set_preference are edited to now use Koha::Config::SysPrefs->set_preference instead of C4::Context->set_preference() One call to delete_preference moved as well
Created attachment 158364 [details] [review] Bug 35250: modify t/db_dependent tests to use Koha::Config::SysPrefs' set_preference instead of C4::Context Some work left to be done, including missing use at top of files
Created attachment 159735 [details] [review] Bug 35250: Use lazy loading to reduce memory footprint of C4/Context.pm This simply removes some 'use' declarations at the top of the file to instead use 'require' later on. The effect can be validated with a simple one-liner (provided KOHA_CONF and PERL5LIB being set) perl -e "use C4::Context; while(1){sleep(1);}" And monitoring the memoring usage in tools like 'top'
Created attachment 159736 [details] [review] Bug 35250: remove usages of Koha::Config::Syspref(s) from C4::Context Use straigth SQL queries to access systempreferences instead of relying on the ORM, thus removing all direct and indirect dependencies to Koha::Object To remove all dependencies between C4::Context and the db object model, we move the calls to set_preference and delete_prefence directly to the class object Koha::Config::SysPrefs C4::Context::preference remains but uses straight SQL C4::Context::set_preference moves to Koha::Config::SysPrefs::set_preference C4::Context::delete_preference moves to Koha::Config::SysPrefs::delete_preference
Created attachment 159737 [details] [review] Bug 35250: modify scripts calling C4::Context::set_preference All the scripts using the call C4::Context->set_preference are edited to now use Koha::Config::SysPrefs->set_preference instead of C4::Context->set_preference() One call to delete_preference moved as well
Created attachment 159738 [details] [review] Bug 35250: modify t/db_dependent tests to use Koha::Config::SysPrefs' set_preference instead of C4::Context Some work left to be done, including missing use at top of files
Rebased
Created attachment 163733 [details] [review] Bug 35250: Use lazy loading to reduce memory footprint of C4/Context.pm This simply removes some 'use' declarations at the top of the file to instead use 'require' later on. The effect can be validated with a simple one-liner (provided KOHA_CONF and PERL5LIB being set) perl -e "use C4::Context; while(1){sleep(1);}" And monitoring the memoring usage in tools like 'top'
Created attachment 163734 [details] [review] Bug 35250: remove usages of Koha::Config::Syspref(s) from C4::Context Use straigth SQL queries to access systempreferences instead of relying on the ORM, thus removing all direct and indirect dependencies to Koha::Object To remove all dependencies between C4::Context and the db object model, we move the calls to set_preference and delete_prefence directly to the class object Koha::Config::SysPrefs C4::Context::preference remains but uses straight SQL C4::Context::set_preference moves to Koha::Config::SysPrefs::set_preference C4::Context::delete_preference moves to Koha::Config::SysPrefs::delete_preference
Created attachment 163735 [details] [review] Bug 35250: modify scripts calling C4::Context::set_preference All the scripts using the call C4::Context->set_preference are edited to now use Koha::Config::SysPrefs->set_preference instead of C4::Context->set_preference() One call to delete_preference moved as well
Created attachment 163736 [details] [review] Bug 35250: modify t/db_dependent tests to use Koha::Config::SysPrefs' set_preference instead of C4::Context Some work left to be done, including missing use at top of files
Replacing use with require is not recommended. IMO the correct fix here would be to move sysprefs code outside of C4::Context.
Sorry, it's what you are actually doing in the other patches. I think that Koha::Config::Syspref should not use C4::Context, so that you could fetch sysprefs without the whole memory stack.
If that's allowed, I'll explore that. I just got back to it (thus the recent rebase) and applied them on our 23.05s to validate nothing unexpected happens (and I need some RES memory back on our servers :-) )
It is very hard to decouple those fully. Config::SysPrefs uses $C4::Context::use_syspref_cache, which is in C4::Context because the sub preference() is still there and might use the cache. Of course, all this could/should be moved to Koha/Config/SysPrefs, but we're talking hundred of calls to change from C4::Context->preference to Koha::Config::SysPrefs->preference(), and that would be hell to rebase repetitively. Right now (locally), I get a 50M footprint to simple use of Koha::Config::SysPrefs, and 21M for C4::Context. I don't think SysPrefs can live without C4/Context because how would it find its DB connection? Somewhere, SysPrefs needs Context. But at least, Context doesn't need SysPrefs and that would make a big difference to a lot of scripts if we could decouple Context completely from the db objects. If I understand right. In short, the only thing I see I could do "more" is moving the preference subs to SysPrefs, but that seems like a task to automate just before a Pushed to Master, not something that can be rebased for months.