Bug 36549

Summary: es_indexer_daemon.pl leaks memory
Product: Koha Reporter: David Cook <dcook>
Component: Architecture, internals, and plumbingAssignee: Bugs List <koha-bugs>
Status: REOPENED --- QA Contact: Testopia <testopia>
Severity: normal    
Priority: P5 - low    
Version: Main   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39769
GIT URL: Change sponsored?: ---
Patch complexity: --- Documentation contact:
Documentation submission: Text to go in the release notes:
Version(s) released in:
Circulation function:

Description David Cook 2024-04-09 02:24:18 UTC
background_jobs_worker.pl runs its tasks in child processes which return their memory to the OS when they complete.

es_indexer_daemon.pl runs everything in itself which means if it needs to allocate a large amount of memory for an indexing job, it will hold on to that memory allocation for its entire process life.

--

I've got an es_indexer_daemon.pl which is using over 1GB of memory. Once it's done with its indexing activity, I'm going to restart it to get it back down to a more reasonable amount of allocated memory. 

I reckon we should be forking off child processes to do the worker. We don't need many. We just need 1 that is short lived so that it can return the memory back to the OS.
Comment 1 David Cook 2024-11-01 01:21:31 UTC
In prod, we seem to restart the services often enough this isn't a problem, so going to close this one for now.
Comment 2 David Cook 2025-02-11 23:44:56 UTC
Re-opening this one as it looks like it's still an issue, and Elasticsearch usage is becoming more prevalent...
Comment 3 David Cook 2025-04-29 02:19:04 UTC
Note also that since es_indexer_daemon.pl doesn't fork workers, it also seems to have a L1 cache issue...