From 88daf6717bde4a20c3eac68cf184f92bdc039cf7 Mon Sep 17 00:00:00 2001
From: Tomas Cohen Arazi <tomascohen@gmail.com>
Date: Wed, 15 Jan 2014 21:06:04 -0300
Subject: [PATCH] Bug 9579: fix truncation of facets containing multi-byte
 characters
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

We seem to be relying on whatever Zoom::Results->render return, and
Perl doesn't explicitly consider it UNICODE data. That's why CORE::substr
(and probably CORE::length too) cut the bytes wrong.

This patch just decodes the UTF-8 data that render() returns and then
Perl behaves, heh.

It uses Encode::decode_utf8 which is already a dependency for the current
stable Koha releases.

REVISED TEST PLAN
-----------------
1) Import the attached sample records.
2) Rebuild your indexes
3) In OPAC search for يكيمكتبات : قبسي ، كرم
-- There will be ugly diamonds with question marks in the facets
4) apply the patch
5) Search again.
-- The names will be properly truncated.
NOTE: This test assumes FacetLabelTruncationLength = 20.

Sponsored-by: Universidad Nacional de Cordoba

Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Works as described, tested with several German, English and
the Arabic test record. Arabic strings now display correctly
and no regression was found.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

I've reviewed it and approve its inclusion in 3.14.x and earlier.  I
will use the patches for bug 11096, once they pass QA, for the master
branch.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>
---
 C4/Search.pm |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/C4/Search.pm b/C4/Search.pm
index 18a0f5f..d09fe4c 100644
--- a/C4/Search.pm
+++ b/C4/Search.pm
@@ -36,6 +36,7 @@ use URI::Escape;
 use Business::ISBN;
 use MARC::Record;
 use MARC::Field;
+use Encode qw/decode_utf8/;
 use utf8;
 use vars qw($VERSION @ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $DEBUG);
 
@@ -503,8 +504,8 @@ sub getRecords {
                       $size > $facets_maxrecs ? $facets_maxrecs : $size;
                     for my $facet (@$facets) {
                         for ( my $j = 0 ; $j < $jmax ; $j++ ) {
-                            my $render_record =
-                              $results[ $i - 1 ]->record($j)->render();
+                            my $render_record = decode_utf8(
+                              $results[ $i - 1 ]->record($j)->render());
                             my @used_datas = ();
                             foreach my $tag ( @{ $facet->{tags} } ) {
 
-- 
1.7.10.4