Bugzilla – Attachment 121561 Details for
Bug 28316
Fix ES crashes related to various punctuation characters
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Help
|
New Account
|
Log In
[x]
|
Forgot Password
Login:
[x]
[patch]
Bug 28316: different QueryRegexEscapeOptions' values work as expected
Bug-28316-different-QueryRegexEscapeOptions-values.patch (text/plain), 4.56 KB, created by
Peter Vashchuk
on 2021-06-03 08:20:35 UTC
(
hide
)
Description:
Bug 28316: different QueryRegexEscapeOptions' values work as expected
Filename:
MIME Type:
Creator:
Peter Vashchuk
Created:
2021-06-03 08:20:35 UTC
Size:
4.56 KB
patch
obsolete
>From 84c9f52881f4d9b2fd38553100c69d51110ce2cc Mon Sep 17 00:00:00 2001 >From: Petro Vashchuk <stalkernoid@gmail.com> >Date: Wed, 2 Jun 2021 17:58:01 +0300 >Subject: [PATCH] Bug 28316: different QueryRegexEscapeOptions' values work as > expected > >This patch ensures that the behavior with >QueryRegexEscapeOptions set to values other than >"Escape" still work as expected. > >It does so by storing the contents of regexes >before escaping square and curly brackets and >then restores the contents of regexes back to how >it was before. > >Also this patch covers this with tests. >--- > .../Elasticsearch/QueryBuilder.pm | 27 ++++++++++++++++--- > .../SearchEngine/Elasticsearch/QueryBuilder.t | 16 ++++++++--- > 2 files changed, 37 insertions(+), 6 deletions(-) > >diff --git a/Koha/SearchEngine/Elasticsearch/QueryBuilder.pm b/Koha/SearchEngine/Elasticsearch/QueryBuilder.pm >index 1bff8a3cba..5b82567a4f 100644 >--- a/Koha/SearchEngine/Elasticsearch/QueryBuilder.pm >+++ b/Koha/SearchEngine/Elasticsearch/QueryBuilder.pm >@@ -943,14 +943,35 @@ sub _clean_search_term { > $term =~ s/(\s+)(:+)$lookahead/$1/g; > $term =~ s/^://; > >+ $term = $self->_query_regex_escape_process($term); >+ > # replace all ranges with any square/curly brackets combinations to temporary substitutions (ex: "{a TO b]"" -> "~~LC~~a TO b~~RS~~") > $term =~ s/(\{|\[)([^\s\[\]\{\}]+ TO [^\s\[\]\{\}]+)(\}|\])/'~~L'.($1 eq '[' ? 'S':'C').'~~'.$2.'~~R'.($3 eq ']' ? 'S':'C').'~~'/ge; >- # screen all brackets with backslash >- $term =~ s/([\{\}\[\]])/\\$1/g; >+ >+ # store the content of regex before escaping brackets: >+ my @saved_regexes; >+ my $rgx_i = 0; >+ while( >+ $term =~ s@( >+ (?<!\\)(?:[\\]{2})*/ >+ (?:[^/]+|(?<=\\)(?:[\\]{2})*/)+ >+ (?<!\\)(?:[\\]{2})*/ >+ )$lookahead@"~~XI$rgx_i~~"@ex >+ ) { >+ @saved_regexes[$rgx_i++] = $1; >+ } >+ >+ $term =~ s/([\{\}\[\]])$lookahead/\\$1/g; >+ >+ # restore the regex content after escaping brackets: >+ for (my $i = 0; $i < @saved_regexes; $i++) { >+ $term =~ s/~~XI$i~~/$saved_regexes[$i]/; >+ } >+ > # restore temporary weird substitutions back to normal brackets > $term =~ s/~~L(C|S)~~([^\s\[\]\{\}]+ TO [^\s\[\]\{\}]+)~~R(C|S)~~/($1 eq 'S' ? '[':'{').$2.($3 eq 'S' ? ']':'}')/ge; > >- $term = $self->_query_regex_escape_process($term); >+ warn Koha::Nug::dumper([$term],['$term']); > > return $term; > } >diff --git a/t/Koha/SearchEngine/Elasticsearch/QueryBuilder.t b/t/Koha/SearchEngine/Elasticsearch/QueryBuilder.t >index 7175f63743..7579170f20 100755 >--- a/t/Koha/SearchEngine/Elasticsearch/QueryBuilder.t >+++ b/t/Koha/SearchEngine/Elasticsearch/QueryBuilder.t >@@ -187,7 +187,7 @@ subtest '_split_query() tests' => sub { > }; > > subtest '_clean_search_term() tests' => sub { >- plan tests => 13; >+ plan tests => 15; > > my $qb; > ok( >@@ -219,12 +219,22 @@ subtest '_clean_search_term() tests' => sub { > $res = $qb->_clean_search_term('test [123 TO 345]'); > is($res, 'test [123 TO 345]', 'keep inculsive range untouched'); > >- $res = $qb->_clean_search_term('test [test TO TEST} [and] {123 TO 456]'); >- is($res, 'test [test TO TEST} \[and\] {123 TO 456]', 'keep exclusive range untouched'); >+ $res = $qb->_clean_search_term('test [test TO TEST} ["[and] {123 TO 456]" "[balanced]"]'); >+ is($res, 'test [test TO TEST} \["[and] {123 TO 456]" "[balanced]"\]', 'keep exclusive range untouched'); > > $res = $qb->_clean_search_term('test[]test TO TEST] [ {123 to 345}}'); > is($res, 'test\[\]test TO TEST\] \[ \{123 to 345\}\}', 'screen all square and curly brackets'); > >+ t::lib::Mocks::mock_preference('QueryRegexEscapeOptions', 'escape'); >+ >+ $res = $qb->_clean_search_term('test inside regexps /this [a-z]/ and \/not [a-z]\/ and that [a-z] [a TO z]'); >+ is($res, 'test inside regexps \/this \[a-z\]\/ and \/not \[a-z\]\/ and that \[a-z\] [a TO z]', 'test inside regexps with QueryRegexEscapeOptions parameter set to escape'); >+ >+ t::lib::Mocks::mock_preference('QueryRegexEscapeOptions', 'dont_escape'); >+ >+ $res = $qb->_clean_search_term('test inside regexps /this [a-z]/ /this2 [a-z]/ [but] /this3 [a-z]/ and \/not [a-z]\/ and that [a-z] [a TO z]'); >+ is($res, 'test inside regexps /this [a-z]/ /this2 [a-z]/ \[but\] /this3 [a-z]/ and \/not \[a-z\]\/ and that \[a-z\] [a TO z]', 'test inside regexps with QueryRegexEscapeOptions parameter set to dont_escape'); >+ > $res = $qb->_clean_search_term('ti:test AND kw:test'); > is($res, 'title:test AND test', 'ti converted to title, kw converted to empty string, dangling colon removed with space preserved'); > >-- >2.31.1
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Diff
|
Splinter Review
Attachments on
bug 28316
:
121435
|
121443
|
121561
|
121562
|
121751
|
121752
|
121753
|
121754
|
121755
|
121756
|
121837
|
121838
|
121839
|
121840
|
121841
|
121842
|
121975
|
121976
|
121977
|
121978
|
121979
|
121980
|
122103
|
122104
|
122105
|
122106
|
122107
|
122108
|
122201
|
122202
|
122203
|
122204
|
122205
|
122206
|
122801
|
122802
|
122803
|
122804
|
122805
|
122806
|
123601
|
123602
|
123603
|
123604
|
123605
|
123606
|
124372
|
124373
|
124374
|
124375
|
124376
|
124377
|
124378
|
124560
|
124561
|
124562
|
124563
|
124564
|
124565
|
124566
|
124709
|
124710
|
124711
|
124712
|
124713
|
124714
|
124715