Attachment 114227 Details for Bug 26996 – Truncate commit size to make sure it's an integer - QA update

[patch] Truncate commit size to make sure it's an integer - QA update

0001-Bug-26996-Convert-Elasticsearch-indexer-commit-buffe.patch (text/plain), 1.59 KB, created by Björn Nylén on 2020-12-07 08:17:46 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Björn Nylén

Created: 2020-12-07 08:17:46 UTC

Size: 1.59 KB

patch

obsolete

>From f5ccbd32e2dfb085180395bfa97d7011760578e4 Mon Sep 17 00:00:00 2001
>From: =?UTF-8?q?Bj=C3=B6rn=20Nyl=C3=A9n?= <bjorn.nylen@ub.lu.se>
>Date: Wed, 11 Nov 2020 08:35:21 +0100
>Subject: [PATCH] Bug 26996: Convert Elasticsearch indexer commit buffer size
> to integer
>
>When multithreaded indexing is used, the commit size for children are spread
>out resulting in them being of type float. When records are processed and the
>commit counter decreased it may then never reach *exactly* 0. This means records
>are never commited. This patch makes sure the counter is an integer to avoid the
>problem.
>
>To test you must find a set of circumstances that causes the issue. For me:
>1. Run: ./rebuild_elasticsearch -v -b -p 2 -c 400
>2. Note that only one process is logging "Committing xxx records..."
>3. Kill processes.
>4. Apply patch.
>5. Repeat 1
>6. Note that both processes are logging "Committing xxx records..."
>
>Sponsored-by: Lund University Library
>---
> misc/search_tools/rebuild_elasticsearch.pl | 1 +
> 1 file changed, 1 insertion(+)
>
>diff --git a/misc/search_tools/rebuild_elasticsearch.pl b/misc/search_tools/rebuild_elasticsearch.pl
>index ff83562f25..6b9487fcb5 100755
>--- a/misc/search_tools/rebuild_elasticsearch.pl
>+++ b/misc/search_tools/rebuild_elasticsearch.pl
>@@ -188,6 +188,7 @@ if ($slice_count > 1) {
>     }
>     # Fudge the commit count a bit to spread out the Elasticsearch commits
>     $commit *= 1 + 0.10 * $slice_index;
>+    $commit = int( $commit );
>     _log(1, "Processing slice @{[$slice_index + 1]} of $slice_count\n");
>     $iterator_options{slice} = { index => $slice_index, count => $slice_count };
> }
>-- 
>2.25.1
>

Actions: View | Diff | Splinter Review

Attachments on bug 26996: 113482 | 114227 | 114232 | 114679