Attachment 80253 Details for Bug 20151 – Bug 20151: always use current language for stemming

[patch] Bug 20151: always use current language for stemming

Bug-20151-always-use-current-language-for-stemming.patch (text/plain), 5.24 KB, created by Martin Renvoize (ashimema) on 2018-10-09 08:31:38 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Martin Renvoize (ashimema)

Created: 2018-10-09 08:31:38 UTC

Size: 5.24 KB

patch

obsolete

>From 1950e52dd883e96bdcc3cc74f0c78bc9ab772c0c Mon Sep 17 00:00:00 2001
>From: Fridolin Somers <fridolin.somers@biblibre.com>
>Date: Wed, 7 Feb 2018 15:06:56 +0100
>Subject: [PATCH] Bug 20151: always use current language for stemming
>
>When stemming is enabled, in catalog searching "C4::Search::_build_stemmed_operand" will transform query operand into stemmed operand using stemmer Lingua::Stem::Snowball with a specified language.
>This stemmer returns undef stemmed operand if no language is defined.
>
>In main catalog search (catalogue/search.pl) current language is used.
>But in other pages "acqui/neworderbiblio.pl" and "cataloguing/addbooks.pl" no language is defined so stemmed operand is empty and so stemming is not applied.
>
>This patch corrects by returning in "C4::Search::_build_stemmed_operand" operand without change if no langage is defined.
>And uses current langage in pages "acqui/neworderbiblio.pl" and "cataloguing/addbooks.pl" so all catalog search uses stemming.
>
>Test plan :
>1) Enable system preferences QueryStemming and QueryWeightFields
>2) Disable system preferences QueryAutoTruncate, QueryFuzzy and UseQueryParser
>3) Go to intranet main page and click on "Search the catalog" tab
>4) Perform a search (without index) that uses the stemming, for example searching for "years" will also match "year"
>5) Note how many results you get, for example "year" gets 24 results and "years" gets 24 results
>6) Go to "Cataloging" module
>7) Perform a search on same word in "Cataloging search" tab
>8) Note how many results you get
>9) Without patch you get fewer results than first search (step 5) because stemming is not applied, for example "year" gets 11 results and "years" gets 15 results
>10) With patch you get the same results as first search (step 5) because stemming is applied, for example "year" and "years" gets 24 results
>11) Same tests in aquisition module
>12) On a basket, click "Add to basket" and perform a search in "From an existing record"
>
>Signed-off-by: David Nind <david@davidnind.com>
>Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
>---
> C4/Search.pm            | 3 +++
> acqui/neworderbiblio.pl | 5 ++++-
> cataloguing/addbooks.pl | 5 ++++-
> 3 files changed, 11 insertions(+), 2 deletions(-)
>
>diff --git a/C4/Search.pm b/C4/Search.pm
>index 06e837de03..46fafcda9d 100644
>--- a/C4/Search.pm
>+++ b/C4/Search.pm
>@@ -968,6 +968,9 @@ sub _build_stemmed_operand {
>     require Lingua::Stem::Snowball ;
>     my $stemmed_operand=q{};
> 
>+    # Stemmer needs language
>+    return $operand unless $lang;
>+
>     # If operand contains a digit, it is almost certainly an identifier, and should
>     # not be stemmed.  This is particularly relevant for ISBNs and ISSNs, which
>     # can contain the letter "X" - for example, _build_stemmend_operand would reduce
>diff --git a/acqui/neworderbiblio.pl b/acqui/neworderbiblio.pl
>index f083398e4f..b48737d79f 100755
>--- a/acqui/neworderbiblio.pl
>+++ b/acqui/neworderbiblio.pl
>@@ -64,6 +64,7 @@ use C4::Auth;
> use C4::Output;
> use C4::Koha;
> use C4::Budgets qw/ GetBudgetHierarchy /;
>+use C4::Languages qw(getlanguage);
> 
> use Koha::Acquisition::Booksellers;
> use Koha::SearchEngine;
>@@ -83,6 +84,7 @@ my $booksellerid     = $params->{'booksellerid'};
> my $basketno         = $params->{'basketno'};
> my $sub              = $params->{'sub'};
> my $bookseller       = Koha::Acquisition::Booksellers->find( $booksellerid );
>+my $lang             = C4::Languages::getlanguage($input);
> 
> # getting the template
> my ( $template, $loggedinuser, $cookie ) = get_template_and_user(
>@@ -106,7 +108,8 @@ my $searcher = Koha::SearchEngine::Search->new({index => $Koha::SearchEngine::BI
> if ($QParser) {
>     $builtquery = $query;
> } else {
>-    ( undef,$builtquery,undef,undef,undef,undef,undef,undef,undef,undef) = $builder->build_query_compat(undef,\@operands);
>+        ( undef, $builtquery, undef, undef, undef, undef, undef, undef, undef, undef ) =
>+          $builder->build_query_compat( undef, \@operands, undef, undef, undef, 0, $lang );
> }
> my ( $error, $marcresults, $total_hits ) = $searcher->simple_search_compat($builtquery, $results_per_page * ($page - 1), $results_per_page);
> 
>diff --git a/cataloguing/addbooks.pl b/cataloguing/addbooks.pl
>index 253683db1d..e374cd33ac 100755
>--- a/cataloguing/addbooks.pl
>+++ b/cataloguing/addbooks.pl
>@@ -32,6 +32,7 @@ use C4::Biblio;
> use C4::Breeding;
> use C4::Output;
> use C4::Koha;
>+use C4::Languages qw(getlanguage);
> use C4::Search;
> 
> use Koha::BiblioFrameworks;
>@@ -46,6 +47,7 @@ my $query   = $input->param('q');
> my @value   = $input->multi_param('value');
> my $page    = $input->param('page') || 1;
> my $results_per_page = 20;
>+my $lang = C4::Languages::getlanguage($input);
> 
> 
> my ( $template, $loggedinuser, $cookie ) = get_template_and_user(
>@@ -75,7 +77,8 @@ if ($query) {
>     if ($QParser) {
>         $builtquery = $query;
>     } else {
>-        ( undef,$builtquery,undef,undef,undef,undef,undef,undef,undef,undef) = $builder->build_query_compat(undef,\@operands);
>+        ( undef, $builtquery, undef, undef, undef, undef, undef, undef, undef, undef ) =
>+          $builder->build_query_compat( undef, \@operands, undef, undef, undef, 0, $lang );
>     }
>     # find results
>     my ( $error, $marcresults, $total_hits ) = $searcher->simple_search_compat($builtquery, $results_per_page * ($page - 1), $results_per_page);
>-- 
>2.19.0

Actions: View | Diff | Splinter Review

Attachments on bug 20151: 71285 | 78994 | 78997 | 80253