Bugzilla – Attachment 92023 Details for
Bug 23204
Add script for exporting Elasticsearch mappings to YAML
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Help
|
New Account
|
Log In
[x]
|
Forgot Password
Login:
[x]
[patch]
Bug 23204: move code in a unit tested sub
Bug-23204-move-code-in-a-unit-tested-sub.patch (text/plain), 8.21 KB, created by
Alex Arnaud
on 2019-08-06 08:04:43 UTC
(
hide
)
Description:
Bug 23204: move code in a unit tested sub
Filename:
MIME Type:
Creator:
Alex Arnaud
Created:
2019-08-06 08:04:43 UTC
Size:
8.21 KB
patch
obsolete
>From 28fef3b995f318143ed2294c68f9945642789704 Mon Sep 17 00:00:00 2001 >From: Alex Arnaud <alex.arnaud@biblibre.com> >Date: Tue, 6 Aug 2019 07:45:39 +0000 >Subject: [PATCH] Bug 23204: move code in a unit tested sub > >--- > Koha/SearchEngine/Elasticsearch.pm | 45 ++++++++ > misc/search_tools/export_elasticsearch_mappings.pl | 33 +----- > .../Koha/SearchEngine/Elasticsearch/ExportConfig.t | 116 +++++++++++++++++++++ > 3 files changed, 164 insertions(+), 30 deletions(-) > create mode 100644 t/db_dependent/Koha/SearchEngine/Elasticsearch/ExportConfig.t > >diff --git a/Koha/SearchEngine/Elasticsearch.pm b/Koha/SearchEngine/Elasticsearch.pm >index 4435015..4f8b062 100644 >--- a/Koha/SearchEngine/Elasticsearch.pm >+++ b/Koha/SearchEngine/Elasticsearch.pm >@@ -241,6 +241,51 @@ sub get_elasticsearch_mappings { > return $all_mappings{$self->index}; > } > >+=head2 raw_elasticsearch_mappings >+ >+Return elasticsearch mapping as it is in database. >+marc_type: marc21|unimarc|normarc >+ >+$raw_mappings = raw_elasticsearch_mappings( $marc_type ) >+ >+=cut >+ >+sub raw_elasticsearch_mappings { >+ my ( $marc_type ) = @_; >+ >+ my $schema = Koha::Database->new()->schema(); >+ >+ my $search_fields = Koha::SearchFields->search(); >+ >+ my $mappings = {}; >+ while ( my $search_field = $search_fields->next ) { >+ >+ my $marc_to_fields = $schema->resultset('SearchMarcToField')->search( { search_field_id => $search_field->id } ); >+ >+ while ( my $marc_to_field = $marc_to_fields->next ) { >+ my $marc_map = Koha::SearchMarcMaps->find( $marc_to_field->search_marc_map_id ); >+ >+ next if $marc_type && $marc_map->marc_type ne $marc_type; >+ >+ $mappings->{ $marc_map->index_name }{ $search_field->name }{label} = $search_field->label; >+ $mappings->{ $marc_map->index_name }{ $search_field->name }{type} = $search_field->type; >+ $mappings->{ $marc_map->index_name }{ $search_field->name }{facet_order} = $search_field->facet_order; >+ >+ push (@{ $mappings->{ $marc_map->index_name }{ $search_field->name }{mappings} }, >+ { >+ facet => $marc_to_field->facet || '', >+ marc_type => $marc_map->marc_type, >+ marc_field => $marc_map->marc_field, >+ sort => $marc_to_field->sort, >+ suggestible => $marc_to_field->suggestible || '' >+ }); >+ >+ } >+ } >+ >+ return $mappings; >+} >+ > =head2 _get_elasticsearch_field_config > > Get the Elasticsearch field config for the given purpose and data type. >diff --git a/misc/search_tools/export_elasticsearch_mappings.pl b/misc/search_tools/export_elasticsearch_mappings.pl >index 1668de5..bd96f81 100644 >--- a/misc/search_tools/export_elasticsearch_mappings.pl >+++ b/misc/search_tools/export_elasticsearch_mappings.pl >@@ -56,6 +56,7 @@ use Modern::Perl; > use Koha::Database; > use Koha::SearchFields; > use Koha::SearchMarcMaps; >+use Koha::SearchEngine::Elasticsearch; > > use YAML; > use Getopt::Long; >@@ -76,34 +77,6 @@ if ( $type && $type !~ /^(marc21|unimarc|normarc)$/ ) { > pod2usage(1); > } > >-my $schema = Koha::Database->new()->schema(); >+my $mappings = Koha::SearchEngine::Elasticsearch::raw_elasticsearch_mappings( $type ); > >-my $search_fields = Koha::SearchFields->search(); >- >-my $yaml = {}; >-while ( my $search_field = $search_fields->next ) { >- >- my $marc_to_fields = $schema->resultset('SearchMarcToField')->search( { search_field_id => $search_field->id } ); >- >- while ( my $marc_to_field = $marc_to_fields->next ) { >- my $marc_map = Koha::SearchMarcMaps->find( $marc_to_field->search_marc_map_id ); >- >- next if $type && $marc_map->marc_type ne $type; >- >- $yaml->{ $marc_map->index_name }{ $search_field->name }{label} = $search_field->label; >- $yaml->{ $marc_map->index_name }{ $search_field->name }{type} = $search_field->type; >- $yaml->{ $marc_map->index_name }{ $search_field->name }{facet_order} = $search_field->facet_order; >- >- push (@{ $yaml->{ $marc_map->index_name }{ $search_field->name }{mappings} }, >- { >- facet => $marc_to_field->facet || '', >- marc_type => $marc_map->marc_type, >- marc_field => $marc_map->marc_field, >- sort => $marc_to_field->sort, >- suggestible => $marc_to_field->suggestible || '' >- }); >- >- } >-} >- >-print Dump($yaml); >+print Dump($mappings); >diff --git a/t/db_dependent/Koha/SearchEngine/Elasticsearch/ExportConfig.t b/t/db_dependent/Koha/SearchEngine/Elasticsearch/ExportConfig.t >new file mode 100644 >index 0000000..4fc461a >--- /dev/null >+++ b/t/db_dependent/Koha/SearchEngine/Elasticsearch/ExportConfig.t >@@ -0,0 +1,116 @@ >+#!/usr/bin/perl >+# >+# This file is part of Koha. >+# >+# Koha is free software; you can redistribute it and/or modify it >+# under the terms of the GNU General Public License as published by >+# the Free Software Foundation; either version 3 of the License, or >+# (at your option) any later version. >+# >+# Koha is distributed in the hope that it will be useful, but >+# WITHOUT ANY WARRANTY; without even the implied warranty of >+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the >+# GNU General Public License for more details. >+# >+# You should have received a copy of the GNU General Public License >+# along with Koha; if not, see <http://www.gnu.org/licenses>. >+ >+use Modern::Perl; >+ >+use Test::More tests => 16; >+ >+use Koha::Database; >+use Koha::SearchFields; >+use Koha::SearchMarcMaps; >+ >+use_ok('Koha::SearchEngine::Elasticsearch'); >+ >+my $schema = Koha::Database->new->schema; >+ >+$schema->storage->txn_begin; >+ >+Koha::SearchFields->search->delete; >+Koha::SearchMarcMaps->search->delete; >+$schema->resultset('SearchMarcToField')->search->delete; >+ >+ >+ >+my $search_field = Koha::SearchFields->find_or_create( >+ { >+ name => 'title', >+ label => 'Title', >+ type => 'string', >+ weight => 17 >+ >+ }, >+ { key => 'name' } ); >+ >+my $marc_field = Koha::SearchMarcMaps->find_or_create( >+ { >+ index_name => 'biblios', >+ marc_type => 'marc21', >+ marc_field => '247' >+ } ); >+ >+$search_field->add_to_search_marc_maps($marc_field, >+ { >+ facet => 0, >+ suggestible => 0, >+ sort => undef >+ } ); >+ >+$marc_field = Koha::SearchMarcMaps->find_or_create( >+ { >+ index_name => 'biblios', >+ marc_type => 'marc21', >+ marc_field => '212' >+ } ); >+ >+$search_field->add_to_search_marc_maps($marc_field, >+ { >+ facet => 0, >+ suggestible => 0, >+ sort => undef >+ } ); >+ >+$marc_field = Koha::SearchMarcMaps->find_or_create( >+ { >+ index_name => 'biblios', >+ marc_type => 'unimarc', >+ marc_field => '200a' >+ } ); >+ >+$search_field->add_to_search_marc_maps($marc_field, >+ { >+ facet => 0, >+ suggestible => 1, >+ sort => undef >+ } ); >+ >+my $mappings = Koha::SearchEngine::Elasticsearch::raw_elasticsearch_mappings(); >+ >+is( $mappings->{biblios}{title}{type}, 'string', 'Title is of type string'); >+is( $mappings->{biblios}{title}{label}, 'Title', 'title has label Title'); >+is( $mappings->{biblios}{title}{facet_order}, undef, 'Facet order is undef'); >+ >+is(scalar(@{ $mappings->{biblios}{title}{mappings} }), 3, 'Title has 3 mappings'); >+ >+my $f247_map = $mappings->{biblios}{title}{mappings}[0]; >+is( $f247_map->{marc_field}, 247, 'First mapping is on field 247'); >+is( $f247_map->{marc_type}, 'marc21', 'First mapping is for marc21'); >+is( $f247_map->{facet}, '', 'First mapping facet is empty'); >+is( $f247_map->{suggestible}, '', 'First mapping is not suggestible'); >+is( $f247_map->{sort}, undef, 'First mapping is not sortable'); >+ >+my $f212_map = $mappings->{biblios}{title}{mappings}[1]; >+is( $f212_map->{marc_field}, 212, 'Second mapping is on field 247'); >+is( $f212_map->{marc_type}, 'marc21', 'Second mapping is for marc21'); >+is( $f212_map->{facet}, '', 'Second mapping facet is empty'); >+is( $f212_map->{suggestible}, '', 'Second mapping is not suggestible'); >+is( $f212_map->{sort}, undef, 'Second mapping is not sortable'); >+ >+$mappings = Koha::SearchEngine::Elasticsearch::raw_elasticsearch_mappings('unimarc'); >+ >+is(scalar(@{ $mappings->{biblios}{title}{mappings} }), 1, 'Title has 1 mappings'); >+ >+$schema->storage->txn_rollback; >-- >2.7.4
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Diff
|
Splinter Review
Attachments on
bug 23204
:
90999
|
91350
|
92022
|
92023
|
92024
|
92174
|
93524
|
95986
|
100960
|
100961
|
100962
|
100963
|
100964
|
100965
|
101742
|
101850
|
101851
|
101852
|
101853
|
101854
|
101855
|
101856
|
101857
|
101858
|
102010