Bugzilla – Attachment 109260 Details for
Bug 17661
Differences in field ending (whitespace, punctuation) cause duplicate facets
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Help
|
New Account
|
Log In
[x]
|
Forgot Password
Login:
[x]
[patch]
Bug 17661: (follow-up) Update regex to support Unicode characters
Bug-17661-follow-up-Update-regex-to-support-Unicod.patch (text/plain), 2.54 KB, created by
Katrin Fischer
on 2020-08-27 20:02:27 UTC
(
hide
)
Description:
Bug 17661: (follow-up) Update regex to support Unicode characters
Filename:
MIME Type:
Creator:
Katrin Fischer
Created:
2020-08-27 20:02:27 UTC
Size:
2.54 KB
patch
obsolete
>From e34290fbb26cbe6bd7ff3009c1a8b4b6e87bf886 Mon Sep 17 00:00:00 2001 >From: Nick Clemens <nick@bywatersolutions.com> >Date: Thu, 27 Aug 2020 16:27:09 +0000 >Subject: [PATCH] Bug 17661: (follow-up) Update regex to support Unicode > characters >MIME-Version: 1.0 >Content-Type: text/plain; charset=UTF-8 >Content-Transfer-Encoding: 8bit > >Rather than limiting initials to [A-Z] we should test for a broad >range of uppercase letters. > >The ES/Zebra changes are slightly different because of Perl vs Java regex >conventions. POerl may support either, but I found 'Uppercase' to be a bit more explicit > >More info here: >https://perldoc.perl.org/perlunicode.html > >TO test: >Same plan as before but use Ð. as the ending initial >Confirm the period is preserved and other punctuation removed > >Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> >--- > C4/Search.pm | 4 ++-- > admin/searchengine/elasticsearch/index_config.yaml | 2 +- > 2 files changed, 3 insertions(+), 3 deletions(-) > >diff --git a/C4/Search.pm b/C4/Search.pm >index e2f2d0936b..d39bbe350f 100644 >--- a/C4/Search.pm >+++ b/C4/Search.pm >@@ -680,7 +680,7 @@ sub _get_facets_data_from_record { > next if $field->indicator(1) eq 'z'; > > my $data = $field->as_string( $subfield_letters, $facet->{ sep } ); >- $data =~ s/\s*(?<![A-Z])[.\-,;]*\s*$//; >+ $data =~ s/\s*(?<!\p{Uppercase})[.\-,;]*\s*$//; > > unless ( grep { $_ eq $data } @used_datas ) { > push @used_datas, $data; >@@ -779,7 +779,7 @@ sub _get_facet_from_result_set { > my $facets = {}; > foreach my $term ( @terms ) { > my $facet_value = $term->textContent; >- $facet_value =~ s/\s*(?<![A-Z])[.\-,;]*\s*$//; >+ $facet_value =~ s/\s*(?<!\p{Uppercase})[.\-,;]*\s*$//; > $facet_value =~ s/\Q$internal_sep\E/$sep/ if defined $sep; > $facets->{ $facet_value } = ( defined $facets->{$facet_value} ) ? $facets->{ $facet_value } + $term->getAttribute( 'occur' ) : $term->getAttribute( 'occur' ); > } >diff --git a/admin/searchengine/elasticsearch/index_config.yaml b/admin/searchengine/elasticsearch/index_config.yaml >index dfae04dd97..e424e92a0b 100644 >--- a/admin/searchengine/elasticsearch/index_config.yaml >+++ b/admin/searchengine/elasticsearch/index_config.yaml >@@ -39,6 +39,6 @@ index: > replacement: '' > facet: > type: pattern_replace >- pattern: '\s*(?<![A-Z])[.\-,;]*\s*$' >+ pattern: '\s*(?<!\p{Lu})[.\-,;]*\s*$' > replacement: '' > index.mapping.total_fields.limit: 10000 >-- >2.11.0
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Diff
|
Splinter Review
Attachments on
bug 17661
:
57658
|
57659
|
68897
|
106637
|
108742
|
109117
|
109242
|
109259
| 109260 |
109368