Sometimes when we run "systemctl restart koha-commmon" the SIP server doesn't get restarted but might just get stopped leading to longer than expected downtime. There is probably some sort of race condition.
I am not sure if this is still valid, but at least there is no mention of SIP in the output of koha-common restart: sudo service koha-common restart Restarting Koha ILS: koha-commonStopping Plack daemon for kohadev:. Stopping Z39.50/SRU daemon for kohadev:. Stopping Koha worker daemon for kohadev (default):. Stopping Koha worker daemon for kohadev (long_tasks):. Stopping Koha indexing daemon for kohadev:. Successfully stopped all services for instance koha-common:. Starting Plack daemon for kohadev:. Starting Z39.50/SRU daemon for kohadev:. Starting Koha worker daemon for kohadev (default):. Starting Koha worker daemon for kohadev (long_tasks):. Starting Koha indexing daemon for kohadev:. .
I've seen this too, I think it happens when there are a lot of SIP workers/processes and it takes some time for them to kill.
This is still valid. Upping verbosity for koha-sip --start reveals the underlying issue. The SIP daemon doesn't stop in time to be able to start again. During koha-common startup, we are checking if SIP is already running. If so, we will not start the SIP server, and it will stay in stopped state until it is manually started again. See line 11 "SIP server already running for ...": > 1 06:07:43.x systemd[1]: Stopping koha-common.service - Start required services for each Koha instance... > 2 06:07:43.x koha-common[x]: * Stopping SIP server for production_instance > 3 06:07:43.x koha-common[x]: ...done. > 4 ... > 5 06:07:44.x systemd[1]: koha-common.service: Deactivated successfully. > 6 06:07:44.x systemd[1]: koha-common.service: Unit process x (daemon) remains running after unit stopped. > 7 06:07:44.x systemd[1]: koha-common.service: Unit process x (es_indexer_daem) remains running after unit stopped. > 8 06:07:44.x systemd[1]: Stopped koha-common.service - Start required services for each Koha instance > 9 ... > 10 06:07:44.x systemd[1]: Starting koha-common.service - Start required services for each Koha instance... > 11 06:07:45.x koha-common[x]: * Warning: SIP server already running for production_instance
Created attachment 188707 [details] [review] Bug 27115: Use start-stop-daemon to stop in koha-sip There has been an issue where a restart of koha-sip would sometimes not bring the SIP server back alive. This is because we have not been waiting for koha-sip --stop to actually stop the SIP server(s). It is then possible for a restart to attempt the --start command while a previous instance of SIP server is still running. As the --start command in this case is not allowed, we would then end up not starting the SIP server(s) at all. This patch replaces daemon --stop with start-stop-daemon --stop to ensure koha-sip daemon actually stops. start-stop-daemon provides us the --retry option, which we are able to utilize in waiting for the stop process to finish. To test: 1. Before applying this patch 2. Run this command export SIPINSTANCE="your_koha_instance_name" You can your_koha_instance_name using the koha-list command 3. Configure a SIP server (add some min_servers, e.g. 6, to SIPConfig.xml) 4. koha-sip --enable $SIPINSTANCE 5. koha-sip --verbose --start $SIPINSTANCE 6. Observe "Starting SIP server for ..." 7. koha-sip --verbose --stop $SIPINSTANCE; ps aux | grep $SIPINSTANCE-koha-sip 8. Observe "Stopping SIP server for instancename" 9. In the output, observe the number of SIP servers you defined in step 3, plus one. 10. koha-sip --verbose --start $SIPINSTANCE 11. Apply this patch, or edit the koha-sip directly to get changes in place in your environment, e.g. using command nano $(which koha-sip) 12. koha-sip --verbose --stop $SIPINSTANCE; ps aux | grep $SIPINSTANCE-koha-sip 13. Observe "Stopping SIP server for instancename" 14. In the output, observe no more SIP servers alive
(In reply to Lari Taskula from comment #3) > > 7 06:07:44.x systemd[1]: koha-common.service: Unit process x (es_indexer_daem) remains running after unit stopped. Btw this line also concerns me. A couple of months ago we had a case where ES indexer was suddenly no longer running, most likely after a restart. Could be related to this issue and perhaps we should use start-stop-daemon for koha-es-indexer --stop as well.