Bugzilla – Attachment 81507 Details for
Bug 18316
Add weighting/relevancy options to ElasticSearch
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Help
|
New Account
|
Log In
[x]
|
Forgot Password
Login:
[x]
[patch]
Bug 18316: Ability to weight search fields
Bug-18316-Ability-to-weight-search-fields.patch (text/plain), 12.25 KB, created by
Martin Renvoize (ashimema)
on 2018-10-29 14:44:22 UTC
(
hide
)
Description:
Bug 18316: Ability to weight search fields
Filename:
MIME Type:
Creator:
Martin Renvoize (ashimema)
Created:
2018-10-29 14:44:22 UTC
Size:
12.25 KB
patch
obsolete
>From 3b12ea03bf18015076ff355d3e93f845392d9638 Mon Sep 17 00:00:00 2001 >From: Alex Arnaud <alex.arnaud@gmail.com> >Date: Thu, 29 Mar 2018 14:20:59 +0000 >Subject: [PATCH] Bug 18316: Ability to weight search fields >MIME-Version: 1.0 >Content-Type: text/plain; charset=UTF-8 >Content-Transfer-Encoding: 8bit > >Weight can be defined between 1 and 99. >only search fields mapped with biblios can be weighted > >Test plan: > - apply this patch, > - update schema file (perl misc/devel/update_dbix_class_files.pl), > - go to Administration > Search engine configuration, > - in search fields tab, define weights for some fields and save, > - check weights has been saved, > - reset some weights (empty) and save, > - check weights has been reset > >Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr> > >Signed-off-by: Nick Clemens <nick@bywatersolutions.com> >Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr> > >Rebased-by: Alex Arnaud <alex.arnaud@biblibre.com> >Signed-off-by: Ere Maijala <ere.maijala@helsinki.fi> >Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> >--- > Koha/SearchField.pm | 45 ++++++ > admin/searchengine/elasticsearch/mappings.pl | 14 +- > .../bug_18316_add-weight-column.perl | 9 ++ > installer/data/mysql/kohastructure.sql | 1 + > .../searchengine/elasticsearch/mappings.tt | 19 +++ > t/db_dependent/Koha/SearchField.t | 135 ++++++++++++++++++ > 6 files changed, 221 insertions(+), 2 deletions(-) > create mode 100644 installer/data/mysql/atomicupdate/bug_18316_add-weight-column.perl > create mode 100644 t/db_dependent/Koha/SearchField.t > >diff --git a/Koha/SearchField.pm b/Koha/SearchField.pm >index 0d92974c26..a734204cba 100644 >--- a/Koha/SearchField.pm >+++ b/Koha/SearchField.pm >@@ -20,6 +20,7 @@ use Modern::Perl; > use Carp; > > use Koha::Database; >+use Koha::SearchMarcMaps; > > use base qw(Koha::Object); > >@@ -38,6 +39,50 @@ sub add_to_search_marc_maps { > return $self->_result()->add_to_search_marc_maps($search_field->_result, $params); > } > >+=head3 search_marc_maps >+ >+my $search_marc_maps = $search_field->search_marc_maps; >+ >+=cut >+ >+sub search_marc_maps { >+ my ( $self ) = @_; >+ >+ my $marc_type = lc C4::Context->preference('marcflavour'); >+ my @marc_maps = (); >+ >+ my $schema = Koha::Database->new->schema; >+ my @marc_map_fields = $schema->resultset('SearchMarcToField')->search({ >+ search_field_id => $self->id >+ }); >+ >+ return @marc_maps unless @marc_map_fields; >+ >+ foreach my $marc_field ( @marc_map_fields ) { >+ my $marc_map = Koha::SearchMarcMaps->find( $marc_field->search_marc_map_id ); >+ push @marc_maps, $marc_map if $marc_map->marc_type eq $marc_type; >+ } >+ >+ return @marc_maps; >+ >+} >+ >+=head3 is_mapped_biblios >+ >+my $is_mapped_biblios = $search_field->is_mapped_biblios >+ >+=cut >+ >+sub is_mapped_biblios { >+ my ( $self ) = @_; >+ >+ foreach my $marc_map ( $self->search_marc_maps ) { >+ return 1 if $marc_map->index_name eq 'biblios'; >+ } >+ >+ return 0; >+} >+ > =head3 type > > =cut >diff --git a/admin/searchengine/elasticsearch/mappings.pl b/admin/searchengine/elasticsearch/mappings.pl >index ecd8741533..8108c1e1cc 100755 >--- a/admin/searchengine/elasticsearch/mappings.pl >+++ b/admin/searchengine/elasticsearch/mappings.pl >@@ -51,6 +51,7 @@ if ( $op eq 'edit' ) { > my @field_name = $input->param('search_field_name'); > my @field_label = $input->param('search_field_label'); > my @field_type = $input->param('search_field_type'); >+ my @field_weight = $input->param('search_field_weight'); > > my @index_name = $input->param('mapping_index_name'); > my @search_field_name = $input->param('mapping_search_field_name'); >@@ -65,9 +66,12 @@ if ( $op eq 'edit' ) { > my $field_name = $field_name[$i]; > my $field_label = $field_label[$i]; > my $field_type = $field_type[$i]; >+ my $field_weight = $field_weight[$i]; >+ > my $search_field = Koha::SearchFields->find( { name => $field_name }, { key => 'name' } ); > $search_field->label($field_label); > $search_field->type($field_type); >+ $search_field->weight($field_weight || ''); > $search_field->store; > } > >@@ -136,8 +140,14 @@ for my $index_name (qw| biblios authorities |) { > push @indexes, { index_name => $index_name, mappings => \@mappings }; > } > >-my $search_fields = $schema->resultset('SearchField')->search; >-my @all_search_fields = $search_fields->search( {}, { order_by => ['name'] } ); >+my $search_fields = Koha::SearchFields->search( {}, { order_by => ['name'] } ); >+my @all_search_fields; >+while ( my $search_field = $search_fields->next ) { >+ my $search_field_unblessed = $search_field->unblessed; >+ $search_field_unblessed->{mapped_biblios} = 1 if $search_field->is_mapped_biblios; >+ push @all_search_fields, $search_field_unblessed; >+} >+ > $template->param( > indexes => \@indexes, > all_search_fields => \@all_search_fields, >diff --git a/installer/data/mysql/atomicupdate/bug_18316_add-weight-column.perl b/installer/data/mysql/atomicupdate/bug_18316_add-weight-column.perl >new file mode 100644 >index 0000000000..185a2b5565 >--- /dev/null >+++ b/installer/data/mysql/atomicupdate/bug_18316_add-weight-column.perl >@@ -0,0 +1,9 @@ >+$DBversion = 'XXX'; >+if( CheckVersion( $DBversion ) ) { >+ if( !column_exists( 'search_field', 'weight' ) ) { >+ $dbh->do( "ALTER TABLE search_field ADD COLUMN weight int" ); >+ } >+ >+ SetVersion( $DBversion ); >+ print "Upgrade to $DBversion done (Bug 18316 - Add column search_field.weight)\n"; >+} >\ No newline at end of file >diff --git a/installer/data/mysql/kohastructure.sql b/installer/data/mysql/kohastructure.sql >index 62c0dcecfb..a1df11d7e2 100644 >--- a/installer/data/mysql/kohastructure.sql >+++ b/installer/data/mysql/kohastructure.sql >@@ -1474,6 +1474,7 @@ CREATE TABLE `search_field` ( > `name` varchar(255) NOT NULL COMMENT 'the name of the field as it will be stored in the search engine', > `label` varchar(255) NOT NULL COMMENT 'the human readable name of the field, for display', > `type` ENUM('', 'string', 'date', 'number', 'boolean', 'sum', 'isbn', 'stdno') NOT NULL COMMENT 'what type of data this holds, relevant when storing it in the search engine', >+ `weight` int default NULL, > PRIMARY KEY (`id`), > UNIQUE KEY (`name` (191)) > ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci; >diff --git a/koha-tmpl/intranet-tmpl/prog/en/modules/admin/searchengine/elasticsearch/mappings.tt b/koha-tmpl/intranet-tmpl/prog/en/modules/admin/searchengine/elasticsearch/mappings.tt >index cc7b28cb85..b3cc2bf9b7 100644 >--- a/koha-tmpl/intranet-tmpl/prog/en/modules/admin/searchengine/elasticsearch/mappings.tt >+++ b/koha-tmpl/intranet-tmpl/prog/en/modules/admin/searchengine/elasticsearch/mappings.tt >@@ -92,6 +92,13 @@ a.add, a.delete { > <h1>Search engine configuration</h1> > <div class="warning"> > Warning: Any changes to the configuration will only take effect after a full reindex. Until then searching may not work correctly. >+ >+ <p>Weight: define weight between 1 and 99.</p> >+ <ol> >+ <li>only search fields mapped with biblios can be weighted</li> >+ <li>search will be done on weighted fields only</li> >+ <li>if no field is weighted, search will be done on all the record</li> >+ </ol> > </div> > [% IF errors %] > <div class="error"> >@@ -141,6 +148,7 @@ a.add, a.delete { > <th>Name</th> > <th>Label</th> > <th>Type</th> >+ <th>Weight</th> > </tr> > </thead> > <tbody> >@@ -190,6 +198,17 @@ a.add, a.delete { > [% END %] > </select> > </td> >+ [% IF search_field.mapped_biblios %] >+ <td> >+ [% IF search_field.weight %] >+ <input type="number" name="search_field_weight" value="[% search_field.weight %]" min="1" max="99" /> >+ [% ELSE %] >+ <input type="number" name="search_field_weight" min="1" max="99" /> >+ [% END %] >+ </td> >+ [% ELSE %] >+ <td><input type="hidden" name="search_field_weight" value=""></td> >+ [% END %] > </tr> > [% END %] > </tbody> >diff --git a/t/db_dependent/Koha/SearchField.t b/t/db_dependent/Koha/SearchField.t >new file mode 100644 >index 0000000000..2e4bed1435 >--- /dev/null >+++ b/t/db_dependent/Koha/SearchField.t >@@ -0,0 +1,135 @@ >+#!/usr/bin/perl >+ >+# Copyright 2018 Biblibre >+# >+# This file is part of Koha >+# >+# Koha is free software; you can redistribute it and/or modify it >+# under the terms of the GNU General Public License as published by >+# the Free Software Foundation; either version 3 of the License, or >+# (at your option) any later version. >+# >+# Koha is distributed in the hope that it will be useful, but >+# WITHOUT ANY WARRANTY; without even the implied warranty of >+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the >+# GNU General Public License for more details. >+# >+# You should have received a copy of the GNU General Public License >+# along with Koha; if not, see <http://www.gnu.org/licenses>. >+ >+use Modern::Perl; >+ >+use Test::More tests => 8; >+ >+use Koha::Database; >+use Koha::SearchFields; >+ >+use t::lib::Mocks; >+use t::lib::TestBuilder; >+ >+my $schema = Koha::Database->new->schema; >+$schema->storage->txn_begin; >+ >+my $builder = t::lib::TestBuilder->new; >+ >+t::lib::Mocks::mock_preference('marcflavour', 'marc21'); >+ >+my $sf = $builder->build({ >+ source => 'SearchField', >+}); >+ >+my $smm = $builder->build({ >+ source => 'SearchMarcMap', >+ value => { >+ index_name => 'biblios', >+ marc_type => 'marc21', >+ marc_field => '410abcdef' >+ } >+}); >+ >+my $smtf = $builder->build({ >+ source => 'SearchMarcToField', >+ value => { >+ search_marc_map_id => $smm->{id}, >+ search_field_id => $sf->{id} >+ } >+}); >+ >+my $smm2 = $builder->build({ >+ source => 'SearchMarcMap', >+ value => { >+ index_name => 'biblios', >+ marc_type => 'unimarc', >+ marc_field => '410abcdef' >+ } >+}); >+ >+my $smtf2 = $builder->build({ >+ source => 'SearchMarcToField', >+ value => { >+ search_marc_map_id => $smm2->{id}, >+ search_field_id => $sf->{id} >+ } >+}); >+ >+my $search_field = Koha::SearchFields->find($sf->{id}); >+my @marc_map = $search_field->search_marc_maps; >+is(scalar(@marc_map), 1, 'search_marc_maps should return 1 marc map'); >+is($marc_map[0]->index_name, 'biblios', 'Marc map index name is biblios'); >+is($marc_map[0]->marc_type, 'marc21', 'Marc map type is marc21'); >+is($marc_map[0]->marc_field, '410abcdef', 'Marc map field is 410abcdef'); >+ >+ok($search_field->is_mapped_biblios, 'Search field 1 is mapped with biblios'); >+ >+my $sf2 = $builder->build({ >+ source => 'SearchField', >+}); >+ >+my $smm3 = $builder->build({ >+ source => 'SearchMarcMap', >+ value => { >+ index_name => 'authorities', >+ marc_type => 'marc21', >+ marc_field => '700a' >+ } >+}); >+ >+my $smtf3 = $builder->build({ >+ source => 'SearchMarcToField', >+ value => { >+ search_marc_map_id => $smm3->{id}, >+ search_field_id => $sf2->{id} >+ } >+}); >+ >+$search_field = Koha::SearchFields->find($sf2->{id}); >+ok(!$search_field->is_mapped_biblios, 'Search field is mapped with authorities only'); >+ >+my $smm4 = $builder->build({ >+ source => 'SearchMarcMap', >+ value => { >+ index_name => 'biblios', >+ marc_type => 'marc21', >+ marc_field => '200a' >+ } >+}); >+ >+my $smtf4 = $builder->build({ >+ source => 'SearchMarcToField', >+ value => { >+ search_marc_map_id => $smm4->{id}, >+ search_field_id => $sf2->{id} >+ } >+}); >+ >+$search_field = Koha::SearchFields->find($sf2->{id}); >+ok($search_field->is_mapped_biblios, 'Search field is mapped with authorities and biblios'); >+ >+my $sf3 = $builder->build({ >+ source => 'SearchField', >+}); >+ >+$search_field = Koha::SearchFields->find($sf3->{id}); >+ok(!$search_field->is_mapped_biblios, 'Search field is not mapped'); >+ >+$schema->storage->txn_rollback; >\ No newline at end of file >-- >2.19.1
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Diff
|
Splinter Review
Attachments on
bug 18316
:
61472
|
73139
|
73140
|
73492
|
73493
|
73600
|
73601
|
74112
|
74115
|
74116
|
74117
|
74714
|
74720
|
74751
|
74752
|
74753
|
74754
|
75073
|
75074
|
75075
|
75076
|
75458
|
75490
|
75519
|
75520
|
75521
|
75522
|
75523
|
75524
|
75525
|
75526
|
75527
|
75528
|
75529
|
75530
|
75531
|
75536
|
75537
|
75538
|
75539
|
75540
|
75541
|
75699
|
75730
|
77661
|
77662
|
77663
|
77664
|
77665
|
77666
|
77667
|
78396
|
78397
|
78398
|
78399
|
78400
|
78401
|
78402
|
78403
|
80278
|
80335
|
81004
|
81005
|
81006
|
81007
|
81008
|
81009
|
81010
|
81011
|
81012
| 81507 |
81508
|
81509
|
81510
|
81511
|
81512
|
81513
|
81514
|
81515
|
81516
|
82050