Attachment 152550 Details for Bug 33270 – Bug 33270: Warn errors but catch exceptions when encountering record issues in OAI-PMH

[patch] Bug 33270: Warn errors but catch exceptions when encountering record issues in OAI-PMH

Bug-33270-Warn-errors-but-catch-exceptions-when-en.patch (text/plain), 3.33 KB, created by Nick Clemens (kidclamp) on 2023-06-22 11:25:45 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Nick Clemens (kidclamp)

Created: 2023-06-22 11:25:45 UTC

Size: 3.33 KB

patch

obsolete

>From a8dcffc6a0e009d5d892844dcb2fc080a3b53a69 Mon Sep 17 00:00:00 2001
>From: Nick Clemens <nick@bywatersolutions.com>
>Date: Thu, 22 Jun 2023 11:22:18 +0000
>Subject: [PATCH] Bug 33270: Warn errors but catch exceptions when encountering
> record issues in OAI-PMH
>
>This patch alters the OAI code to attempt to load a badly encoded record by removing non XML characters. OAI harvests are used to keep external catalogs or display layers in sync with Koha. We should not break this connection when there is a bad record, but should allow the harvest to complete - warn the errors, and rely on Koha warnings/errors/tools to idnetify and fix the problem on the Koha side
>
>Test plan, assumes using KTD default data - otherwise you need to find and import a record with encoding issues:
>1 - Enable OAI-PMH system preference
>2 - Browse to:
>    http://localhost:8080/cgi-bin/koha/oai.pl?verb=ListRecords&resumptionToken=marcxml/350////0/0/352
>3 - 500 error:
>    Invalid data, cannot decode metadata object (biblio_metadata.id=368, biblionumber=369, format=marcxml, schema=MARC21, decoding_error=':8: parser error : PCDATA invalid Char value 31...
>4 - Apply patch, restart all
>5 - Reload the page
>6 - It loads!
>7 - Click 'Metadata' for record 369 - it succeeds!
>8 - Check the logs - confirm you see a warning of the record problem
>---
> Koha/OAI/Server/Repository.pm | 29 ++++++++++++++++++++++++++---
> 1 file changed, 26 insertions(+), 3 deletions(-)
>
>diff --git a/Koha/OAI/Server/Repository.pm b/Koha/OAI/Server/Repository.pm
>index 0cd0df5dcf..16ceec411f 100644
>--- a/Koha/OAI/Server/Repository.pm
>+++ b/Koha/OAI/Server/Repository.pm
>@@ -37,9 +37,12 @@ use YAML::XS;
> use CGI qw/:standard -oldstyle_urls/;
> use C4::Context;
> use C4::Biblio qw( GetFrameworkCode );
>+use C4::Charset qw( StripNonXmlChars );
> use Koha::XSLT::Base;
> use Koha::Biblios;
> 
>+use MARC::Record;
>+
> =head1 NAME
> 
> Koha::OAI::Server::Repository - Handles OAI-PMH requests for a Koha database.
>@@ -178,8 +181,29 @@ sub get_biblio_marcxml {
>     }
> 
>     my $biblio = Koha::Biblios->find($biblionumber);
>-    my $record = $biblio->metadata->record({ embed_items => $with_items, opac => 1 });
>-    if ( $expanded_avs ) {
>+    my $record;
>+    # Koha::Biblio::Metadata->record throws an exception on bad encoding,
>+    # we don't want OAI harvests to die, so we catch and warn, and try to clean the record
>+    eval { $record = $biblio->metadata->record({ embed_items => $with_items, opac => 1 }); };
>+    if ($@) {
>+        my $marcxml_error = $@;
>+        chomp $marcxml_error;
>+        warn $marcxml_error;
>+        eval {
>+            $record = MARC::Record->new_from_xml(
>+                StripNonXmlChars( $biblio->metadata->metadata ), 'UTF-8',
>+                $biblio->metadata->schema
>+            );
>+        };
>+        # If the record still cannot be built, we will warn and return undef
>+        # and record will be reported as deleted
>+        if( $@ ){
>+            $marcxml_error = $@;
>+            chomp $marcxml_error;
>+            warn $marcxml_error;
>+        }
>+    }
>+    if ( $record && $expanded_avs ) {
>         my $frameworkcode = GetFrameworkCode($biblionumber) || '';
>         my $record_processor = Koha::RecordProcessor->new(
>             {
>@@ -192,7 +216,6 @@ sub get_biblio_marcxml {
>         );
>         $record_processor->process($record);
>     }
>-
>     return $record ? $record->as_xml_record() : undef;
> }
> 
>-- 
>2.30.2

Actions: View | Diff | Splinter Review

Attachments on bug 33270: 152550 | 152573 | 152649 | 152650 | 152836 | 152837 | 153325 | 153542 | 153543 | 153544 | 153545 | 153546