Bug 28411 - Investigate memory footprint of the different dependencies
Summary: Investigate memory footprint of the different dependencies
Status: In Discussion
Alias: None
Product: Koha
Classification: Unclassified
Component: Architecture, internals, and plumbing (show other bugs)
Version: Main
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Bugs List
QA Contact: Testopia
URL:
Keywords:
Depends on: 35250
Blocks: 28410
  Show dependency treegraph
 
Reported: 2021-05-21 11:51 UTC by Jonathan Druart
Modified: 2023-11-07 10:45 UTC (History)
2 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
Bug 28411: [DO NOT PUSH] memory_usage.pl (1.29 KB, patch)
2021-05-21 11:54 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 28411: memory_usage.pl output (4.59 KB, patch)
2021-05-21 11:54 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 28411: Use lazy loading to reduce memory footprint of C4/Context.pm (2.35 KB, patch)
2023-10-26 20:24 UTC, Blou
Details | Diff | Splinter Review
Bug 28411: remove Koha::Config::Syspref(s) from C4::Context (3.78 KB, patch)
2023-10-26 20:24 UTC, Blou
Details | Diff | Splinter Review
Bug 28411: Use lazy loading to reduce memory footprint of C4/Context.pm (2.35 KB, patch)
2023-10-27 21:26 UTC, Blou
Details | Diff | Splinter Review
Bug 28411: remove Koha::Config::Syspref(s) from C4::Context (7.46 KB, patch)
2023-10-27 21:26 UTC, Blou
Details | Diff | Splinter Review
Bug 28411: modify scripts' call to set_preference now in Koha::Config::SysPrefs (10.43 KB, patch)
2023-10-27 21:26 UTC, Blou
Details | Diff | Splinter Review
Bug 28411: modify t/db_dependent tests to use Koha::Config::SysPrefs' set_preference insteand of C4::Context (28.53 KB, patch)
2023-10-27 21:26 UTC, Blou
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan Druart 2021-05-21 11:51:27 UTC
In order to know which modules is costing lot of RAM it would be great to have a list the modules with their memory impact.
Comment 1 Jonathan Druart 2021-05-21 11:54:06 UTC
Created attachment 121271 [details] [review]
Bug 28411: [DO NOT PUSH] memory_usage.pl

apt install libmemory-usage-perl
cd /kohadevbox/koha
perl memory_usage.pl|sort -n
Comment 2 Jonathan Druart 2021-05-21 11:54:10 UTC
Created attachment 121272 [details] [review]
Bug 28411: memory_usage.pl output
Comment 3 Blou 2023-10-26 20:24:37 UTC
Created attachment 157956 [details] [review]
Bug 28411: Use lazy loading to reduce memory footprint of C4/Context.pm

This simply removes some 'use' declarations at the top of the file to
instead use 'require' later on.

The effect can be validated with a simple one-liner (provided KOHA_CONF
and PERL5LIB being set)

perl -e "use C4::Context; while(1){sleep(1);}"

And monitoring the memoring usage in tools like 'top'
Comment 4 Blou 2023-10-26 20:24:39 UTC
Created attachment 157957 [details] [review]
Bug 28411: remove Koha::Config::Syspref(s) from C4::Context

Use straigth SQL queries to access systempreferences instead of relying
on the ORM, thus removing all direct and indirect dependencies to
Koha::Object
Comment 5 Blou 2023-10-26 20:25:41 UTC
Added two exploratory patches.  Huge effect on memory footprint, but doesn't pass tests.
Comment 6 Blou 2023-10-27 21:26:35 UTC
Created attachment 158003 [details] [review]
Bug 28411: Use lazy loading to reduce memory footprint of C4/Context.pm

This simply removes some 'use' declarations at the top of the file to
instead use 'require' later on.

The effect can be validated with a simple one-liner (provided KOHA_CONF
and PERL5LIB being set)

perl -e "use C4::Context; while(1){sleep(1);}"

And monitoring the memoring usage in tools like 'top'
Comment 7 Blou 2023-10-27 21:26:37 UTC
Created attachment 158004 [details] [review]
Bug 28411: remove Koha::Config::Syspref(s) from C4::Context

Use straigth SQL queries to access systempreferences instead of relying
on the ORM, thus removing all direct and indirect dependencies to
Koha::Object

To remove all dependencies between C4::Context and the db object model,
we move the calls to set_preference and delete_prefence directly to the
class object Koha::Config::SysPrefs

C4::Context::preference remains but uses straight SQL
C4::Context::set_preference moves to Koha::Config::SysPrefs::set_preference
C4::Context::delete_preference moves to Koha::Config::SysPrefs::delete_preference
Comment 8 Blou 2023-10-27 21:26:39 UTC
Created attachment 158005 [details] [review]
Bug 28411: modify scripts' call to set_preference now in Koha::Config::SysPrefs

All the scripts using the call C4::Context->set_preference are edited to
now use Koha::Config::SysPrefs->set_preference instead of C4::Context->set_preference()

One call to delete_preference moved as well
Comment 9 Blou 2023-10-27 21:26:41 UTC
Created attachment 158006 [details] [review]
Bug 28411: modify t/db_dependent tests to use Koha::Config::SysPrefs' set_preference insteand of C4::Context

Some work left to be done, including missing use at top of files