Bug 36721 - Preload core libraries to speed up worker startup and reduce memory usage
Summary: Preload core libraries to speed up worker startup and reduce memory usage
Status: In Discussion
Alias: None
Product: Koha
Classification: Unclassified
Component: Architecture, internals, and plumbing (show other bugs)
Version: unspecified
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Bugs List
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-04-30 01:50 UTC by David Cook
Modified: 2024-05-22 06:46 UTC (History)
1 user (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
Bug 36721: Add -M Koha::Preload to preload libraries into starman master (2.55 KB, patch)
2024-04-30 06:31 UTC, David Cook
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description David Cook 2024-04-30 01:50:38 UTC
If we preload core libraries using Starman's -M option, we can in theory speed up worker startup and reduce memory usage.
Comment 1 David Cook 2024-04-30 01:57:17 UTC
I've done some experimenting by using a "-M Koha::Preload" in /usr/sbin/koha-plack

You can observe memory savings using "free -m" with the most obvious memory savings being over a large number of workers. 

Speed... if just preloading Plack and Mojolicious libraries, the speed difference is negligible. 

The biggest savings in time/memory I've noticed so far is preloading Koha::Schema which preloads all our 272 DBIx::Class schema files. 

--

If we want to preload other Koha files, I think we'll need to be careful to make sure we're not causing any variable scoping problems.
Comment 2 David Cook 2024-04-30 02:41:48 UTC Comment hidden (obsolete)
Comment 3 David Cook 2024-04-30 06:31:32 UTC
Created attachment 165826 [details] [review]
Bug 36721: Add -M Koha::Preload to preload libraries into starman master

By preloading common libraries into starman master, we save memory
as starman master will share the memory addresses with its starman worker
processes.

We will also save time because each individual worker won't need to load
these common modules. They'll already be done!

To test:
0. Don't apply the patch yet
1. In koha-conf.xml, set the plack workers to a high number
2. Restart Plack
3. Note the memory usage using "free -m"

4. Apply the patch
5. Restart Plack using "debian/scripts/koha-plack"
6. Note the memory usage using "free -m"
7. Memory usage should be significantly less (around 1GB with 16 workers)

8. Comparing time is more difficult, but can be done before/after patch
using strace, top, or by modifying Koha/REST/V1.pm to output a timestamp
at the end of its load. Other ideas are also welcome.
Comment 4 David Cook 2024-04-30 06:33:36 UTC
I don't have benchmarks available at the moment, especially as I was trying out a variety of things.

But with 16 workers I was able to load Koha about 2x faster when using "-M Koha::Preload".

I think resolving bug 36722 will be another big time/speed win...
Comment 5 David Cook 2024-04-30 07:07:43 UTC
(Note: I preload an entire Catalyst app in mod_perl, and it makes for lightning fast worker startup, as all the slow stuff is done in the master process before forking.)
Comment 6 David Cook 2024-05-22 06:46:47 UTC
Note that in https://gitlab.com/koha-community/koha-testing-docker/-/merge_requests/497#note_1915557285 Victor and I talk a bit about using --preload-app with Starman to preload the entire app.

The warnings against doing this on Starman documentation make me reluctant to do that, but we could investigate that too.

Plackup seems to preload the whole app, so if we use that in dev and don't have any issues... it would be tempting to start preloading the app in prod as well, because it would likely have good performance benefits.