Attachment 145173 Details for Bug 32481 – Bug 32481: Limit prefetch size for background jobs worker

[patch] Bug 32481: Limit prefetch size for background jobs worker

Bug-32481-Limit-prefetch-size-for-background-jobs-.patch (text/plain), 1.84 KB, created by Jonathan Druart on 2023-01-10 13:31:49 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Jonathan Druart

Created: 2023-01-10 13:31:49 UTC

Size: 1.84 KB

patch

obsolete

>From d98f6b694dffa155b3d1cd174be2903ea00209ab Mon Sep 17 00:00:00 2001
>From: David Cook <dcook@prosentient.com.au>
>Date: Thu, 22 Dec 2022 22:17:04 +0000
>Subject: [PATCH] Bug 32481: Limit prefetch size for background jobs worker
>
>This patch adds a prefetch size of 1 to the background jobs worker,
>so that it fetches 1 message at a time. Without this change,
>the RabbitMQ connection timeout when too many messages for slow tasks
>are fetched at the same time.
>
>To test:
>0. Apply patch
>1. Run background worker
>2. Rapidly enqueue multiple jobs that in total will take longer
>than 30 minutes to process
>
>Bug 32481: Use correct prefetch syntax for RabbitMQ
>
>According to https://www.rabbitmq.com/stomp.html the header to
>use for managing the prefetch is "prefetch-count".
>
>You can verify the number of delivered and unacknowledged messages
>on a channel on a connection by running "rabbitmqctl list_channels"
>on the RabbitMQ host. This will tell you how many messages have been
>delivered and are awaiting acknowledgement
>
>Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
>---
> misc/background_jobs_worker.pl | 6 +++++-
> 1 file changed, 5 insertions(+), 1 deletion(-)
>
>diff --git a/misc/background_jobs_worker.pl b/misc/background_jobs_worker.pl
>index eb36f34f964..a9eda516813 100755
>--- a/misc/background_jobs_worker.pl
>+++ b/misc/background_jobs_worker.pl
>@@ -78,7 +78,11 @@ if ( $conn ) {
>     # FIXME cf note in Koha::BackgroundJob about $namespace
>     my $namespace = C4::Context->config('memcached_namespace');
>     for my $queue (@queues) {
>-        $conn->subscribe({ destination => sprintf("/queue/%s-%s", $namespace, $queue), ack => 'client' });
>+        $conn->subscribe({
>+            destination => sprintf("/queue/%s-%s", $namespace, $queue),
>+            ack => 'client',
>+            'prefetch-count' => 1,
>+        });
>     }
> }
> while (1) {
>-- 
>2.25.1

Actions: View | Diff | Splinter Review

Attachments on bug 32481: 144809 | 145082 | 145134 | 145173 | 145208