Bugzilla – Attachment 79013 Details for
Bug 20244
Elasticsearch - Indexing improvements
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Help
|
New Account
|
Log In
[x]
|
Forgot Password
Login:
[x]
[patch]
Bug 20244: Add alt script indexing and fix sort field indexing
Bug-20244-Add-alt-script-indexing-and-fix-sort-fie.patch (text/plain), 5.14 KB, created by
Brendan Gallagher
on 2018-09-16 21:44:46 UTC
(
hide
)
Description:
Bug 20244: Add alt script indexing and fix sort field indexing
Filename:
MIME Type:
Creator:
Brendan Gallagher
Created:
2018-09-16 21:44:46 UTC
Size:
5.14 KB
patch
obsolete
>From 650d1f22d03be61e64bf3c3af3d0dad612dac62b Mon Sep 17 00:00:00 2001 >From: Ere Maijala <ere.maijala@helsinki.fi> >Date: Mon, 2 Jul 2018 16:04:05 +0300 >Subject: [PATCH] Bug 20244: Add alt script indexing and fix sort field > indexing > >https://bugs.koha-community.org/show_bug.cgi?id=20244 > >Test plan: > >1. Add a record with alternate script fields in 880 (sample attached in the bug). >2. Make sure the the record can be found e.g. with the alternate script title. >3. Add a record with multiple authors. >4. Check that in the index there is only a single author__sort field. > >Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com> >--- > Koha/SearchEngine/Elasticsearch.pm | 34 +++++++++++++++++++++++++++++----- > t/Koha/SearchEngine/Elasticsearch.t | 8 ++++---- > 2 files changed, 33 insertions(+), 9 deletions(-) > >diff --git a/Koha/SearchEngine/Elasticsearch.pm b/Koha/SearchEngine/Elasticsearch.pm >index dd26f6c..eab4c43 100644 >--- a/Koha/SearchEngine/Elasticsearch.pm >+++ b/Koha/SearchEngine/Elasticsearch.pm >@@ -313,7 +313,7 @@ sub marc_records_to_documents { > my @record_documents; > > sub _process_mappings { >- my ($mappings, $data, $record_document) = @_; >+ my ($mappings, $data, $record_document, $altscript) = @_; > foreach my $mapping (@{$mappings}) { > my ($target, $options) = @{$mapping}; > # Copy (scalar) data since can have multiple targets >@@ -333,23 +333,47 @@ sub marc_records_to_documents { > $options->{property} => $_data > } > } >- push @{$record_document->{$target}}, $_data; >+ >+ # For sort fields, index only a single field with concatenated values >+ # and ignore alt script fields >+ if ($target =~ /__sort$/) { >+ if (!$altscript) { >+ if (!@{$record_document->{$target}}) { >+ push @{$record_document->{$target}}, $_data; >+ } else { >+ @{$record_document->{$target}}[0] .= " $_data"; >+ } >+ } >+ } else { >+ push @{$record_document->{$target}}, $_data; >+ } > } > } > foreach my $record (@{$records}) { > my $record_document = {}; > my $mappings = $rules->{leader}; > if ($mappings) { >- _process_mappings($mappings, $record->leader(), $record_document); >+ _process_mappings($mappings, $record->leader(), $record_document, 0); > } > foreach my $field ($record->fields()) { >- if($field->is_control_field()) { >+ if ($field->is_control_field()) { > my $mappings = $control_fields_rules->{$field->tag()}; > if ($mappings) { >- _process_mappings($mappings, $field->data(), $record_document); >+ _process_mappings($mappings, $field->data(), $record_document, 0); > } > } > else { >+ my $tag = $field->tag(); >+ # Handle alternate scripts in MARC 21 >+ my $altscript = 0; >+ if ($marcflavour eq 'marc21' && $tag eq '880') { >+ my $sub6 = $field->subfield('6'); >+ if ($sub6 =~ /^(...)-\d+/) { >+ $tag = $1; >+ $altscript = 1; >+ } >+ } >+ > my $data_field_rules = $data_fields_rules->{$field->tag()}; > > if ($data_field_rules) { >diff --git a/t/Koha/SearchEngine/Elasticsearch.t b/t/Koha/SearchEngine/Elasticsearch.t >index bb12143..054887c 100644 >--- a/t/Koha/SearchEngine/Elasticsearch.t >+++ b/t/Koha/SearchEngine/Elasticsearch.t >@@ -267,11 +267,11 @@ subtest 'Koha::SearchEngine::Elasticsearch::marc_records_to_documents () tests' > is(scalar @{$docs->[0][1]->{author}}, 2, 'First document author field should contain two values'); > is_deeply($docs->[0][1]->{author}, ['Author 1', 'Corp Author'], 'First document author field should be set correctly'); > >- is(scalar @{$docs->[0][1]->{author__sort}}, 2, 'First document author__sort field should have two values'); >- is_deeply($docs->[0][1]->{author__sort}, ['Author 1', 'Corp Author'], 'First document author__sort field should be set correctly'); >+ is(scalar @{$docs->[0][1]->{author__sort}}, 1, 'First document author__sort field should have a single value'); >+ is_deeply($docs->[0][1]->{author__sort}, ['Author 1 Corp Author'], 'First document author__sort field should be set correctly'); > >- is(scalar @{$docs->[0][1]->{title__sort}}, 3, 'First document title__sort field should have three values'); >- is_deeply($docs->[0][1]->{title__sort}, ['Title:', 'first record', 'Title: first record'], 'First document title__sort field should be set correctly'); >+ is(scalar @{$docs->[0][1]->{title__sort}}, 1, 'First document title__sort field should have a single'); >+ is_deeply($docs->[0][1]->{title__sort}, ['Title: first record Title: first record'], 'First document title__sort field should be set correctly'); > > is(scalar @{$docs->[0][1]->{author__suggestion}}, 2, 'First document author__suggestion field should contain two values'); > is_deeply( >-- >2.1.4
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Diff
|
Splinter Review
Attachments on
bug 20244
:
71919
|
73372
|
73373
|
74321
|
76646
|
76647
|
76648
|
76649
|
78727
|
78728
|
78729
|
79012
|
79013
|
79014
|
82160
|
82161
|
82162
|
82272
|
82273