Attachment 174542 Details for Bug 38416 – Bug 38416: Failover to MARCXML if cannot roundtrip USMARC during indexing

[patch] Bug 38416: Failover to MARCXML if cannot roundtrip USMARC during indexing

Bug-38416-Failover-to-MARCXML-if-cannot-roundtrip-.patch (text/plain), 2.60 KB, created by Marcel de Rooy on 2024-11-15 07:59:28 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Marcel de Rooy

Created: 2024-11-15 07:59:28 UTC

Size: 2.60 KB

patch

obsolete

>From 86afa2ae8b75842363548af9261306a6741f950b Mon Sep 17 00:00:00 2001
>From: David Cook <dcook@prosentient.com.au>
>Date: Mon, 11 Nov 2024 04:30:06 +0000
>Subject: [PATCH] Bug 38416: Failover to MARCXML if cannot roundtrip USMARC
> during indexing
>Content-Type: text/plain; charset=utf-8
>
>This change failsover to MARCXML from USMARC if there are any
>warnings generated by MARC::File::USMARC::decode when trying to
>roundtrip the record.
>
>Test plan:
>0. Apply the patch
>1. Setup your koha-testing-docker to use Elasticsearch
>2. Create a new record with 15,000 characters in the 500$a field
>3. Index that record
>(e.g. perl misc/search_tools/rebuild_elasticsearch.pl --biblios -v -v)
>4. Note that a warning saying the following appears:
>"Warnings encountered while roundtripping a MARC record to/from USMARC.
>Failing over to MARCXML"
>5. View the "Elasticsearch record" on the detail page and note that the
>marc_format is MARCXML
>6. Perform a search for the record (the keyword should be something that
>brings up other results too)
>7. Note that the record appears correctly in the search results
>
>Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
>
>Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
>---
> Koha/SearchEngine/Elasticsearch.pm | 14 +++++++++++++-
> 1 file changed, 13 insertions(+), 1 deletion(-)
>
>diff --git a/Koha/SearchEngine/Elasticsearch.pm b/Koha/SearchEngine/Elasticsearch.pm
>index 5ef5235720..13541d0849 100644
>--- a/Koha/SearchEngine/Elasticsearch.pm
>+++ b/Koha/SearchEngine/Elasticsearch.pm
>@@ -815,7 +815,19 @@ sub marc_records_to_documents {
>                 local $SIG{__WARN__} = sub {
>                     push @warnings, $_[0];
>                 };
>-                $record_document->{'marc_data'} = encode_base64(encode('UTF-8', $record->as_usmarc()));
>+                my $usmarc_record = $record->as_usmarc();
>+
>+                #NOTE: Try to round-trip the record to prove it will work for retrieval after searching
>+                my $decoded_usmarc_record = MARC::Record->new_from_usmarc($usmarc_record);
>+                if ($decoded_usmarc_record->warnings()){
>+                    #NOTE: We override the warnings since they're many and misleading
>+                    @warnings = (
>+                        "Warnings encountered while roundtripping a MARC record to/from USMARC. Failing over to MARCXML.",
>+                    );
>+                }
>+
>+                my $marc_data = encode_base64(encode('UTF-8', $usmarc_record));
>+                $record_document->{'marc_data'} = $marc_data;
>             }
>             if (@warnings) {
>                 # Suppress warnings if record length exceeded
>-- 
>2.39.5

Actions: View | Diff | Splinter Review

Attachments on bug 38416: 174318 | 174319 | 174389 | 174459 | 174460 | 174461 | 174542 | 174543 | 174544