Attachment 167141 Details for Bug 36947 – Bug 36947: Remove diacritics from decomposed strings in ES search

[patch] Bug 36947: Remove diacritics from decomposed strings in ES search

Bug-36947-Remove-diacritics-from-decomposed-string.patch (text/plain), 2.06 KB, created by David Cook on 2024-05-24 04:34:59 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: David Cook

Created: 2024-05-24 04:34:59 UTC

Size: 2.06 KB

patch

obsolete

>From 0257c59b0e9f7685d85d17609f91b957b22fd95c Mon Sep 17 00:00:00 2001
>From: David Cook <dcook@prosentient.com.au>
>Date: Fri, 24 May 2024 04:26:41 +0000
>Subject: [PATCH] Bug 36947: Remove diacritics from decomposed strings in ES
> search
>MIME-Version: 1.0
>Content-Type: text/plain; charset=UTF-8
>Content-Transfer-Encoding: 8bit
>
>This patch removes the diacritics from strings decomposed using
>Unicode::Normalize's NFKD function. It's necessary to provide
>equivalent sorting of accented and unaccented characters.
>
>Test plan:
>0. Apply patch
>1. koha-plack --restart kohadev
>2. Switch to using Elasticsearch and reindex
>koha-elasticsearch -b -v --rebuild kohadev
>3. Setup some test records with authors with accented and unaccented names,
>and different cases for the lead letter
>e.g. Aa author, Ãa author2, aa author3
>4 Do a test search
>e.g. http://localhost:8081/cgi-bin/koha/catalogue/search.pl?q=test
>5. Confirm the facet names are sorted in ascending order regardless
>of accent or case
>e.g.
>Aa author
>Ãa author2
>aa author3
>Farley, David
>Humble, Jez
>---
> Koha/SearchEngine/Elasticsearch/Search.pm | 9 ++++++++-
> 1 file changed, 8 insertions(+), 1 deletion(-)
>
>diff --git a/Koha/SearchEngine/Elasticsearch/Search.pm b/Koha/SearchEngine/Elasticsearch/Search.pm
>index 40ec4ceeb7..f9c2066b20 100644
>--- a/Koha/SearchEngine/Elasticsearch/Search.pm
>+++ b/Koha/SearchEngine/Elasticsearch/Search.pm
>@@ -513,7 +513,7 @@ sub _convert_facets {
>         }
>         if( C4::Context->preference('FacetOrder') eq 'Alphabetical' ){
>             @{ $facet->{facets} } =
>-                sort { NFKD(uc($a->{facet_label_value})) cmp NFKD(uc($b->{facet_label_value})) } @{ $facet->{facets} };
>+                sort { _normalize_str($a->{facet_label_value}) cmp _normalize_str($b->{facet_label_value}) } @{ $facet->{facets} };
>         }
>         push @facets, $facet if exists $facet->{facets};
>     }
>@@ -586,4 +586,11 @@ sub _aggregation_scan {
>     return (undef, \%result, undef);
> }
> 
>+sub _normalize_str {
>+    my ($str) = @_;
>+    my $text = NFKD(uc($str));
>+    $text =~ s/\p{M}//g;
>+    return $text;
>+}
>+
> 1;
>-- 
>2.39.2

Actions: View | Diff | Splinter Review

Attachments on bug 36947: 167110 | 167125 | 167141 | 167142 | 167143 | 167145 | 167146 | 167188 | 167198 | 167880 | 168488 | 170489