From d4cbb20f0037f519c817257b35926bfa414901d1 Mon Sep 17 00:00:00 2001 From: David Cook Date: Tue, 26 May 2020 12:30:53 +1000 Subject: [PATCH] Bug 17842: UTF-8 encode ISO2709 MARC download from cart MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The cart was outputing ISO2709 MARC records with Latin-1 encoding. Records containing non-latin1 characters were automatically re-encoded as UTF-8 browsers, which led to inconsistent character encodings for downloaded MARC files. This patch explicitly encodes ISO2709 MARC characters from the cart download as UTF-8 encoded bytes, which resolves the problem. Test Plan: 0) Don't apply patch 1) Create bib record with only ASCII characters 2) Add a ü character to the title 3) Save bib record 4) Download bib record from cart (opac and staff client) 5) Using xxd or some other program, note that the ü is represented by a FC byte (latin-1 encoded) 6) Apply the patch 7) Download bib record from cart (opac and staff client) 8) Using xxd or some other program, note that the ü is represented by C3 BC bytes (utf-8 encoded) 9) Success (Note that you could potentially use Notepad++ or some other program to open the downloaded file and just note the encoding that it finds. You could also try "chardetect" instead. Lots of options for figuring out the encoding.) --- basket/downloadcart.pl | 5 +++++ opac/opac-downloadcart.pl | 5 +++++ 2 files changed, 10 insertions(+) diff --git a/basket/downloadcart.pl b/basket/downloadcart.pl index 46534493d1..063941fc62 100755 --- a/basket/downloadcart.pl +++ b/basket/downloadcart.pl @@ -72,6 +72,11 @@ if ($bib_list && $format) { if ($format eq 'iso2709') { $output .= $record->as_usmarc(); + if ($output){ + #NOTE: If we don't explicitly UTF-8 encode the output, + #the browser will guess the encoding, and it won't always choose UTF-8. + $output = encode("UTF-8", $output); + } } elsif ($format eq 'ris') { $output .= marc2ris($record); diff --git a/opac/opac-downloadcart.pl b/opac/opac-downloadcart.pl index ec5e2c8e4a..7d07cd0c9f 100755 --- a/opac/opac-downloadcart.pl +++ b/opac/opac-downloadcart.pl @@ -91,6 +91,11 @@ if ($bib_list && $format) { if ($format eq 'iso2709') { $output .= $record->as_usmarc(); + if ($output){ + #NOTE: If we don't explicitly UTF-8 encode the output, + #the browser will guess the encoding, and it won't always choose UTF-8. + $output = encode("UTF-8", $output); + } } elsif ($format eq 'ris') { $output .= marc2ris($record); -- 2.16.4