Bug 41506

Summary: Should we remove RabbitMQ?
Product: Koha Reporter: Katrin Fischer <katrin.fischer>
Component: Architecture, internals, and plumbingAssignee: Bugs List <koha-bugs>
Status: In Discussion --- QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: a.roussos, cornelius.hertfelder, davewood, dcook, emmi.takkinen, fridolin.somers, koha, kyle, m.de.rooy, nick, oliver.behnke, philippe.blouin, tomascohen, victor
Version: Main   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35920
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36484
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=1993
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36702
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39314
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39277
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35092
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31616
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35655
GIT URL: Initiative type: ---
Sponsorship status: --- Comma delimited list of Sponsors:
Crowdfunding goal: 0 Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:

Description Katrin Fischer 2025-12-31 12:51:14 UTC
At the hackfest in Wellington this came up as a question in the discussion and it was decided to bring it up at the next development meeting. 

https://koha-hedgedoc.servers.llownd.net/Development_IRC_meeting_26_November_2025#

The discussion is difficult and we want to include as many opinions/people as possible. I felt it would be a bit lost on bug 35920, so proposing a separate bug report instead.
Comment 1 Katrin Fischer 2025-12-31 12:56:53 UTC
Adding some more personal notes:

* When we had a survey at the hackfest in Marseille last year it showed that almost all of the big providers have deactivated RabbitMQ in their installations. That certainly impacts the amount of testing and development work in that area.

* Currently RabbitMQ is a dependency that is installed with the Koha packages. So currently it requires some hacking to make sure it's not installed and running even when the new pref JobsNotificationMethod is set to SQL polling. 

I'll link open RabbitMQ related bugs in the see also section.
Comment 2 Oliver Behnke 2026-01-08 15:38:11 UTC
Yes, please, even if it's only because of this:

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39314#c10

Cheers
Comment 3 Kyle M Hall (khall) 2026-01-12 12:14:03 UTC
I agree it should be removed. It was a great idea in theory, but in practice it simply doesn't work well.
Comment 4 Tomás Cohen Arazi (tcohen) 2026-01-12 13:39:00 UTC
I'm in favor of removing the use of rabbit (I submitted such patch on bug 35920 already) and we can always rethink the architecture and reintroduce with a better split of concerns (e.g. adding a task manager/scheduler, etc).
Comment 5 Emmi Takkinen 2026-01-13 09:53:30 UTC
I agree with removing this. We have never seen benefit of using it since background jobs seem to work smoothly even without it. So it's main purpose has been to fill our test logs with warnings of not founding Stomp server.
Comment 6 David Cook 2026-01-13 23:01:58 UTC
I'm on the fence (surprise surprise). I don't think that I feel strongly one way or another anymore, but I've got some thoughts (not quite pros/cons) below if you want to read them.

--

I haven't used the SQL polling method for the background workers, so I don't know how effective/efficient it is or how well it supports multiple workers for 1 queue, but I have heard that a lot of people like it. I also have used SQL polling for background jobs in the past and it was good enough for simple use cases.

Originally, I had an interest in RabbitMQ for high performance background jobs, as I was targeting background jobs that needed to be actioned extremely quickly (ie near real-time) and needed a FIFO queue that could ensure proper distribution of tasks to multiple concurrent workers. However, the use case for that was never realised in Koha, so it didn't matter in the end. 

Despite a few hiccups, RabbitMQ has worked pretty well for me in Koha (and on other systems). I think there have been times where people have avoided RabbitMQ rather than engaging with it for fixing bugs. 

But... I think that's also a sign that maybe RabbitMQ isn't a good fit for the Koha community. If an alternative solution has grown up in reaction to it, maybe we need to listen to that. There's also the issue with RabbitMQ not being very easy to upgrade. 

We don't have a lot of resources in the Koha community, and supporting multiple options can be difficult. We already do it with Zebra/Elasticsearch and that's hard enough. 

Anyway, maybe I'll leave it there for now. Basically, I think RabbitMQ provides us with a lot of power and potential to carry Koha into the future. However, I also acknowledge that a lot of people don't like it and don't want it in Koha, and maybe it's overkill for what we need in Koha at present. 

When making choices on technology, I think we do need to think about what we can collectively maintain as a community. It does seem like we have more human resources for maintaining the SQL polling option than the RabbitMQ option.
Comment 7 Andreas Roussos 2026-01-25 20:26:25 UTC
IMHO, to make an informed decision about this issue it'd also be
very interesting to get a rough idea about the number of Libraries
that have switched to SQL polling instead of the default setting
which uses RabbitMQ (i.e. 'STOMP').

We don't (yet) share the JobsNotificationMethod setting with Hea,
but Bug 23849 aims to change that.

It might also be worth correlating the number of Libraries that
use/don't use RabbitMQ with other metrics such as the number of
bibliographic/authority records, the number of patrons, etc. as
this could provide an estimate for how much a Library's size
affects the decision of whether or not to use RabbitMQ.

Just my €0.02 ;-)
Comment 8 Katrin Fischer 2026-01-25 20:53:16 UTC
(In reply to Andreas Roussos from comment #7)
> IMHO, to make an informed decision about this issue it'd also be
> very interesting to get a rough idea about the number of Libraries
> that have switched to SQL polling instead of the default setting
> which uses RabbitMQ (i.e. 'STOMP').

We did a survey at the last Hackfest in Marseille and all the big support providers have turned RabbitMQ off... so that gives some indication. 

> We don't (yet) share the JobsNotificationMethod setting with Hea,
> but Bug 23849 aims to change that.
> 
> It might also be worth correlating the number of Libraries that
> use/don't use RabbitMQ with other metrics such as the number of
> bibliographic/authority records, the number of patrons, etc. as
> this could provide an estimate for how much a Library's size
> affects the decision of whether or not to use RabbitMQ.
> 
> Just my €0.02 ;-)

I know that KIT has deactivated RabbitMQ and we too, so we can tell it's working really well without for libraries with a million records and more.
Comment 9 Emmi Takkinen 2026-01-26 11:23:07 UTC
(In reply to Andreas Roussos from comment #7)
> It might also be worth correlating the number of Libraries that
> use/don't use RabbitMQ with other metrics such as the number of
> bibliographic/authority records, the number of patrons, etc. as
> this could provide an estimate for how much a Library's size
> affects the decision of whether or not to use RabbitMQ.
Our biggest consortium, Vaski libraries, has over 2,5 million bibliographic records, around 235900 users and 74 libraries (all the libraries use same Koha installation). Overall, most of our consortiums have over 1 million bibliographic records and there are total number of 400 libraries in consortiums.
Comment 10 Katrin Fischer 2026-01-26 15:26:20 UTC
I think the question is not if libraries are using it, because I am sure there are libraries using it as it installs by defeault and couldn't be "properly" turned off before the introduction of the system preference only somewhat recently.

The things I worry about are:
* added complexity
* will to maintain and develop
* dependency and upgrade issues
* lack of benefit 

If the SQL polling does the job and it looks like most have moved this direction now, even the real big libraries, then we might want to focus on that solution.
Comment 11 Oliver Behnke 2026-01-30 09:40:35 UTC
A first backwards-compatible change could be to remove rabbitmq-server from the dependencies of the koha-common Debian package. Move it to the list of recommended packages instead, which are installed by default but can also be uninstalled. That would match the JobsNotificationMethod option introduced in bug 35655 on the package level.

Unfortunately, none of the other packages (e.g. koha-core, still experimental) can currently easily replace koha-common without rabbitmq-server, so the change above would be ideal, I think.

Thanks
Comment 12 davewood 2026-02-10 09:56:25 UTC
I would like to see rabbitmq-server moved to the suggested packages in koha-common.
Comment 13 Mark Hofstetter 2026-02-11 08:04:43 UTC
I think it should be removed completly, maybe in a first step just make SQL polling the default, and move rabbitmq-server to the suggested packages in koha-common ( as suggested by David)
Comment 14 Blou 2026-02-18 15:40:05 UTC
Morituri te salutant
Comment 15 Marcel de Rooy 2026-02-19 07:04:11 UTC
I never had a problem with RabbitMQ but the pressure is obvious here. So lets remove it ;)
Comment 16 Fridolin Somers 2026-02-19 13:09:22 UTC
We at BibLibre use SQL Pooling on all instances.

But we have only one Koha (and one DB) per virtual machine so maybe this context makes it easier.
I must say there was sometimes performance issues when Elastiseach didn't had its own deamon.

Now with 24.11 it is fine. Plus our hardware is very powerful so background are started less than a few seconds after creation.