Bug 27115 - Restarting koha-common fails to restart SIP2 server
Summary: Restarting koha-common fails to restart SIP2 server
Status: Signed Off
Alias: None
Product: Koha
Classification: Unclassified
Component: Packaging (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal
Assignee: Bugs List
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-11-30 08:01 UTC by Joonas Kylmälä
Modified: 2025-12-29 18:52 UTC (History)
5 users (show)

See Also:
GIT URL:
Initiative type: ---
Sponsorship status: ---
Comma delimited list of Sponsors:
Crowdfunding goal: 0
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
This fixes an issue when stopping and restarting SIP servers using `koha-sip` - it would sometimes not restart the SIP servers. This was because a restart could attempt the --start command while a previous SIP server was still running. This could result in the SIP server not restarting at all. The fix replaces `daemon --stop` with `start-stop-daemon --stop` in the code, to ensure that all the running SIP servers are actually stopped before restarting.
Version(s) released in:
Circulation function:


Attachments
Bug 27115: Use start-stop-daemon to stop in koha-sip (2.96 KB, patch)
2025-10-30 22:09 UTC, Lari Taskula
Details | Diff | Splinter Review
Bug 27115: Use start-stop-daemon to stop in koha-sip (3.00 KB, patch)
2025-12-29 18:30 UTC, David Nind
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Joonas Kylmälä 2020-11-30 08:01:55 UTC
Sometimes when we run "systemctl restart koha-commmon" the SIP server doesn't get restarted but might just get stopped leading to longer than expected downtime. There is probably some sort of race condition.
Comment 1 Katrin Fischer 2023-07-01 14:24:26 UTC
I am not sure if this is still valid, but at least there is no mention of SIP in the output of koha-common restart:

sudo service koha-common restart
Restarting Koha ILS: koha-commonStopping Plack daemon for kohadev:.
Stopping Z39.50/SRU daemon for kohadev:.
Stopping Koha worker daemon for kohadev (default):.
Stopping Koha worker daemon for kohadev (long_tasks):.
Stopping Koha indexing daemon for kohadev:.
Successfully stopped all services for instance koha-common:.
Starting Plack daemon for kohadev:.
Starting Z39.50/SRU daemon for kohadev:.
Starting Koha worker daemon for kohadev (default):.
Starting Koha worker daemon for kohadev (long_tasks):.
Starting Koha indexing daemon for kohadev:.
.
Comment 2 Liz Rea 2024-08-05 15:30:13 UTC
I've seen this too, I think it happens when there are a lot of SIP workers/processes and it takes some time for them to kill.
Comment 3 Lari Taskula 2025-10-30 09:23:11 UTC
This is still valid.

Upping verbosity for koha-sip --start reveals the underlying issue. The SIP daemon doesn't stop in time to be able to start again. During koha-common startup, we are checking if SIP is already running. If so, we will not start the SIP server, and it will stay in stopped state until it is manually started again.

See line 11 "SIP server already running for ...":

> 1 06:07:43.x systemd[1]: Stopping koha-common.service - Start required services for each Koha instance...
> 2 06:07:43.x koha-common[x]:  * Stopping SIP server for production_instance
> 3 06:07:43.x koha-common[x]:    ...done.
> 4 ...
> 5 06:07:44.x systemd[1]: koha-common.service: Deactivated successfully.
> 6 06:07:44.x systemd[1]: koha-common.service: Unit process x (daemon) remains running after unit stopped.
> 7 06:07:44.x systemd[1]: koha-common.service: Unit process x (es_indexer_daem) remains running after unit stopped.
> 8 06:07:44.x systemd[1]: Stopped koha-common.service - Start required services for each Koha instance
> 9 ...
> 10 06:07:44.x systemd[1]: Starting koha-common.service - Start required services for each Koha instance...
> 11 06:07:45.x koha-common[x]:  * Warning: SIP server already running for production_instance
Comment 4 Lari Taskula 2025-10-30 22:09:09 UTC
Created attachment 188707 [details] [review]
Bug 27115: Use start-stop-daemon to stop in koha-sip

There has been an issue where a restart of koha-sip would sometimes not bring
the SIP server back alive. This is because we have not been waiting for
koha-sip --stop to actually stop the SIP server(s). It is then possible for a
restart to attempt the --start command while a previous instance of SIP server
is still running. As the --start command in this case is not allowed, we would
then end up not starting the SIP server(s) at all.

This patch replaces daemon --stop with start-stop-daemon --stop to ensure
koha-sip daemon actually stops.

start-stop-daemon provides us the --retry option, which we are able to utilize
in waiting for the stop process to finish.

To test:
1. Before applying this patch
2. Run this command

export SIPINSTANCE="your_koha_instance_name"

You can your_koha_instance_name using the koha-list command

3. Configure a SIP server (add some min_servers, e.g. 6, to SIPConfig.xml)
4. koha-sip --enable $SIPINSTANCE
5. koha-sip --verbose --start $SIPINSTANCE
6. Observe "Starting SIP server for ..."
7. koha-sip --verbose --stop $SIPINSTANCE; ps aux | grep $SIPINSTANCE-koha-sip
8. Observe "Stopping SIP server for instancename"
9. In the output, observe the number of SIP servers you defined in step 3, plus
   one.
10. koha-sip --verbose --start $SIPINSTANCE
11. Apply this patch, or edit the koha-sip directly to get changes in place
in your environment, e.g. using command

nano $(which koha-sip)

12. koha-sip --verbose --stop $SIPINSTANCE; ps aux | grep $SIPINSTANCE-koha-sip
13. Observe "Stopping SIP server for instancename"
14. In the output, observe no more SIP servers alive
Comment 5 Lari Taskula 2025-10-30 22:16:18 UTC
(In reply to Lari Taskula from comment #3)
> > 7 06:07:44.x systemd[1]: koha-common.service: Unit process x (es_indexer_daem) remains running after unit stopped.

Btw this line also concerns me. A couple of months ago we had a case where ES indexer was suddenly no longer running, most likely after a restart. Could be related to this issue and perhaps we should use start-stop-daemon for koha-es-indexer --stop as well.
Comment 6 David Nind 2025-12-29 18:30:59 UTC
Created attachment 190766 [details] [review]
Bug 27115: Use start-stop-daemon to stop in koha-sip

There has been an issue where a restart of koha-sip would sometimes not bring
the SIP server back alive. This is because we have not been waiting for
koha-sip --stop to actually stop the SIP server(s). It is then possible for a
restart to attempt the --start command while a previous instance of SIP server
is still running. As the --start command in this case is not allowed, we would
then end up not starting the SIP server(s) at all.

This patch replaces daemon --stop with start-stop-daemon --stop to ensure
koha-sip daemon actually stops.

start-stop-daemon provides us the --retry option, which we are able to utilize
in waiting for the stop process to finish.

To test:
1. Before applying this patch
2. Run this command

export SIPINSTANCE="your_koha_instance_name"

You can your_koha_instance_name using the koha-list command

3. Configure a SIP server (add some min_servers, e.g. 6, to SIPConfig.xml)
4. koha-sip --enable $SIPINSTANCE
5. koha-sip --verbose --start $SIPINSTANCE
6. Observe "Starting SIP server for ..."
7. koha-sip --verbose --stop $SIPINSTANCE; ps aux | grep $SIPINSTANCE-koha-sip
8. Observe "Stopping SIP server for instancename"
9. In the output, observe the number of SIP servers you defined in step 3, plus
   one.
10. koha-sip --verbose --start $SIPINSTANCE
11. Apply this patch, or edit the koha-sip directly to get changes in place
in your environment, e.g. using command

nano $(which koha-sip)

12. koha-sip --verbose --stop $SIPINSTANCE; ps aux | grep $SIPINSTANCE-koha-sip
13. Observe "Stopping SIP server for instancename"
14. In the output, observe no more SIP servers alive

Signed-off-by: David Nind <david@davidnind.com>
Comment 7 David Nind 2025-12-29 18:52:42 UTC
I think I tested this correctly, and that I observed the expected results!

In KTD, the SIP server is already running, but I went through the process of stopping it and then disabling it before working through the test plan.

I've attempted a release note.

Testing notes (using KTD):

1. SIP is already running in KTD for kohadev:
     sudo koha-sip --verbose --status kohadev
     SIP server running for kohadev:.

2. Make the change in step 3 of the test plan - made min_servers = 6: vi /etc/koha/sites/kohadev/SIPconfig.xml

3. Stop the SIP server: 
     sudo koha-sip --verbose --stop kohadev
     Stopping SIP server for kohadev:.

4. Disable the SIP server:
     sudo koha-sip --verbose --disable kohadev
     Stopping running SIP
     Warning: SIP server not running for kohadev:.
     Information: SIP server disabled for kohadev

5. Check that no SIP servers are running:

     ps aux | grep kohadev-koha-sip
     kohadev+   17009  0.0  0.0   9008  2176 pts/0    S+   18:07   0:00 grep kohadev-koha-sip
     
     sudo koha-sip --verbose --status kohadev
     SIP server not running for kohadev: failed!

6. Enable the SIP server:
     sudo koha-sip --verbose --enable kohadev
     Enabling SIP server for kohadev - edit /etc/koha/sites/kohadev/SIPconfig.xml to configure

7. Start the SIP server:
     sudo koha-sip --verbose --start kohadev
     Starting SIP server for kohadev:.

8. Check the number of servers running - this is 7 for me (min_servers = 6):

ps aux | grep $SIPINSTANCE-koha-sip
kohadev+   17366  0.0  0.0   8176  1808 ?        S    18:12   0:00 daemon --name=kohadev-koha-sip --errlog=/var/log/koha/kohadev/sip-error.log --output=/var/log/koha/kohadev/sip-output.log --verbose=1 --respawn --delay=30 --pidfiles=/var/run/koha/kohadev --user=kohadev-koha.kohadev-koha -- perl /kohadevbox/koha/C4/SIP/SIPServer.pm /etc/koha/sites/kohadev/SIPconfig.xml
kohadev+   17368  0.8  0.6 308364 208892 ?       S    18:12   0:01 kohadev-koha-sip: perl /kohadevbox/koha/C4/SIP/SIPServer.pm /etc/koha/sites/kohadev/SIPconfig.xml
kohadev+   17373  0.0  0.5 308364 187492 ?       S    18:12   0:00 kohadev-koha-sip: perl /kohadevbox/koha/C4/SIP/SIPServer.pm /etc/koha/sites/kohadev/SIPconfig.xml
kohadev+   17374  0.0  0.5 308364 187492 ?       S    18:12   0:00 kohadev-koha-sip: perl /kohadevbox/koha/C4/SIP/SIPServer.pm /etc/koha/sites/kohadev/SIPconfig.xml
kohadev+   17375  0.0  0.5 308364 187492 ?       S    18:12   0:00 kohadev-koha-sip: perl /kohadevbox/koha/C4/SIP/SIPServer.pm /etc/koha/sites/kohadev/SIPconfig.xml
kohadev+   17376  0.0  0.5 308364 187492 ?       S    18:12   0:00 kohadev-koha-sip: perl /kohadevbox/koha/C4/SIP/SIPServer.pm /etc/koha/sites/kohadev/SIPconfig.xml
kohadev+   17377  0.0  0.5 308364 187492 ?       S    18:12   0:00 kohadev-koha-sip: perl /kohadevbox/koha/C4/SIP/SIPServer.pm /etc/koha/sites/kohadev/SIPconfig.xml
kohadev+   17378  0.0  0.5 308364 187492 ?       S    18:12   0:00 kohadev-koha-sip: perl /kohadevbox/koha/C4/SIP/SIPServer.pm /etc/koha/sites/kohadev/SIPconfig.xml
kohadev+   17517  0.0  0.0   9008  2304 pts/0    S+   18:14   0:00 grep kohadev-koha-sip

9. Back to step 7 of the test plan:

sudo koha-sip --verbose --stop kohadev; ps aux | grep kohadev-koha-sip
Stopping SIP server for kohadev:.
kohadev+   17366  0.0  0.0   8176  1808 ?        S    18:12   0:00 daemon --name=kohadev-koha-sip --errlog=/var/log/koha/kohadev/sip-error.log --output=/var/log/koha/kohadev/sip-output.log --verbose=1 --respawn --delay=30 --pidfiles=/var/run/koha/kohadev --user=kohadev-koha.kohadev-koha -- perl /kohadevbox/koha/C4/SIP/SIPServer.pm /etc/koha/sites/kohadev/SIPconfig.xml
kohadev+   17368  0.4  0.6 308364 208892 ?       R    18:12   0:01 kohadev-koha-sip: perl /kohadevbox/koha/C4/SIP/SIPServer.pm /etc/koha/sites/kohadev/SIPconfig.xml
kohadev+   17373  0.0  0.5 308364 188900 ?       R    18:12   0:00 kohadev-koha-sip: perl /kohadevbox/koha/C4/SIP/SIPServer.pm /etc/koha/sites/kohadev/SIPconfig.xml
kohadev+   17374  0.0  0.5 308364 188900 ?       R    18:12   0:00 kohadev-koha-sip: perl /kohadevbox/koha/C4/SIP/SIPServer.pm /etc/koha/sites/kohadev/SIPconfig.xml
kohadev+   17375  0.0  0.5 308364 188900 ?       R    18:12   0:00 kohadev-koha-sip: perl /kohadevbox/koha/C4/SIP/SIPServer.pm /etc/koha/sites/kohadev/SIPconfig.xml
kohadev+   17376  0.0  0.5 308364 188900 ?       R    18:12   0:00 kohadev-koha-sip: perl /kohadevbox/koha/C4/SIP/SIPServer.pm /etc/koha/sites/kohadev/SIPconfig.xml
kohadev+   17377  0.0  0.5 308364 188900 ?       R    18:12   0:00 kohadev-koha-sip: perl /kohadevbox/koha/C4/SIP/SIPServer.pm /etc/koha/sites/kohadev/SIPconfig.xml
kohadev+   17378  0.0  0.5 308364 188900 ?       R    18:12   0:00 kohadev-koha-sip: perl /kohadevbox/koha/C4/SIP/SIPServer.pm /etc/koha/sites/kohadev/SIPconfig.xml
kohadev+   17666  0.0  0.0   9008  2048 pts/0    S+   18:16   0:00 grep kohadev-koha-sip

10. Step 10 of the test plan - start the SIP server (7 running after this):
      sudo koha-sip --verbose --start kohadev
      Starting SIP server for kohadev:.

11. After applying the patch - end result is 7 SIP servers running (from e ps aux command): reset_all + step 2 above + restart_all

12. Steps 12 to 14 of the test plan:
      sudo koha-sip --verbose --stop kohadev; ps aux | grep kohadev-koha-sip
      Stopping SIP server for kohadev:.
      kohadev+   20224  0.0  0.0   9008  2048 pts/0    S+   18:28   0:00 grep kohadev-koha-sip