Bug 21534 - ElasticSearch - Wildcards not being analyzed
Summary: ElasticSearch - Wildcards not being analyzed
Status: RESOLVED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching - Elasticsearch (show other bugs)
Version: master
Hardware: All All
: P5 - low normal (vote)
Assignee: Fridolin SOMERS
QA Contact:
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-10-10 12:55 UTC by Marjorie Barry-Vila
Modified: 2019-10-14 20:03 UTC (History)
7 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Trivial patch
Who signed the patch off:
Text to go in the release notes:
Version(s) released in:
19.11.00,19.05.01


Attachments
Bug 21534: Always compute wildcards (1.39 KB, patch)
2019-05-29 13:46 UTC, Fridolin SOMERS
Details | Diff | Splinter Review
Bug 21534: Always compute wildcards (1.44 KB, patch)
2019-05-30 20:21 UTC, Michal Denar
Details | Diff | Splinter Review
Update release notes for the 19.05.00 release (279.43 KB, patch)
2019-05-31 19:08 UTC, Arthur Bousquet
Details | Diff | Splinter Review
Bug 21534: Always compute wildcards (1.49 KB, patch)
2019-05-31 20:03 UTC, Arthur Bousquet
Details | Diff | Splinter Review
Bug 21534: Always compute wildcards (1.55 KB, patch)
2019-06-14 17:57 UTC, Nick Clemens
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Marjorie Barry-Vila 2018-10-10 12:55:47 UTC
Hi,

The following search "Pierrette Saints-Anges" gives no results.
url is opac-search.pl?q=pierrette+Saints-Anges

The search "Pierrette saints anges" works.
url is opac-search.pl?q=pierrette+Saints+Anges

I think dash (-) in my first search is considered like a boolean operator, right?

Has this problem been identified for ElasticSearch?

Regards,
Marjorie
Comment 1 Nick Clemens 2019-02-14 16:04:04 UTC
I tested this on master and cannot recreate.

According to the docs:
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html

Hyphen is considered a boolean 'not' operator

in testing though it seems to work as long as there is no space
Comment 2 Marjorie Barry-Vila 2019-03-26 18:06:52 UTC
(In reply to Nick Clemens from comment #1)
> I tested this on master and cannot recreate.
> 
> According to the docs:
> https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-
> query-string-query.html
> 
> Hyphen is considered a boolean 'not' operator
> 
> in testing though it seems to work as long as there is no space

Hi Nick,
Do you think my bug is a duplicate of https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20334

Regards
Marjorie
Comment 3 Nick Clemens 2019-03-27 11:50:56 UTC
I do (In reply to Marjorie Barry-Vila from comment #2)
> (In reply to Nick Clemens from comment #1)
> > I tested this on master and cannot recreate.
> > 
> > According to the docs:
> > https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-
> > query-string-query.html
> > 
> > Hyphen is considered a boolean 'not' operator
> > 
> > in testing though it seems to work as long as there is no space
> 
> Hi Nick,
> Do you think my bug is a duplicate of
> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20334
> 
> Regards
> Marjorie

I do, if you do too you can mark it as a duplicate
Comment 4 Marjorie Barry-Vila 2019-04-26 15:44:53 UTC
(In reply to Nick Clemens from comment #1)
> I tested this on master and cannot recreate.
> 
> According to the docs:
> https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-
> query-string-query.html
> 
> Hyphen is considered a boolean 'not' operator
> 
> in testing though it seems to work as long as there is no space

Hi Nick,
You may not have been able to reproduce the problem on your side because the QueryAutoTruncate system preference was already set to "only if a * is added"?
I realized that I was getting results if I changed the system preference for this option.

On the other hand, with the preference system on "only if a * is added" if I search :
saints-anges*

this gives no result

(cgi-bin/koha/opac-search.pl?q=saints-anges*)

Could you test this point?

Regards,
Marjorie
Comment 5 Fridolin SOMERS 2019-05-29 13:08:33 UTC
> Hyphen is considered a boolean 'not' operator
I think it is only with a space before : -new
In this case it is an hyphen between 2 works : saints-anges
Comment 6 Fridolin SOMERS 2019-05-29 13:17:27 UTC
Looks like its same problem as Bug 22997, by default wildcards are not used.
Comment 7 Fridolin SOMERS 2019-05-29 13:46:08 UTC
Created attachment 90187 [details] [review]
Bug 21534: Always compute wildcards

You get no results when searching with an hyphen + with * in query string (or with preference QueryAutoTruncate) :
ie /cgi-bin/koha/opac-search.pl?q=saints-anges*

Looks like query-string by default does not compute wildcards, see analyze_wildcard in :
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html

Test plan :
1) Use Elasticsearch
2) Create a record with "saints-anges"
3) Search for "saints-anges" => you get results
4) Search for "saints-anges*" => you get results
5) Search for "saints-ang*" => you get results
Comment 8 Michal Denar 2019-05-30 20:21:03 UTC
Created attachment 90207 [details] [review]
Bug 21534: Always compute wildcards

You get no results when searching with an hyphen + with * in query string (or with preference QueryAutoTruncate) :
ie /cgi-bin/koha/opac-search.pl?q=saints-anges*

Looks like query-string by default does not compute wildcards, see analyze_wildcard in :
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html

Test plan :
1) Use Elasticsearch
2) Create a record with "saints-anges"
3) Search for "saints-anges" => you get results
4) Search for "saints-anges*" => you get results
5) Search for "saints-ang*" => you get results

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 9 Arthur Bousquet 2019-05-31 19:08:18 UTC
Created attachment 90247 [details] [review]
Update release notes for the 19.05.00 release

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

https://bugs.koha-community.org/show_bug.cgi?id=11573
Comment 10 Katrin Fischer 2019-05-31 19:25:23 UTC
Hi Arthus, same issue as on bug 11573 here, wrong patch attached. Can you please check your git bz command? If the patch is applied, the right command here would be: git bz attach 21534 HEAD

For multiple patches (4): git bz attach xxxx HEAD~4..
Comment 11 Katrin Fischer 2019-05-31 19:26:09 UTC
(In reply to Katrin Fischer from comment #10)
> Hi Arthus, same issue as on bug 11573 here, wrong patch attached. Can you
> please check your git bz command? If the patch is applied, the right command
> here would be: git bz attach 21534 HEAD
> 
> For multiple patches (4): git bz attach xxxx HEAD~4..

Arthur, not Arthus, sorry!
Comment 12 Arthur Bousquet 2019-05-31 19:27:29 UTC
Yes, sorry it really seems like I'm having some trouble with git-bz =/
Comment 13 Katrin Fischer 2019-05-31 19:29:23 UTC
No problem, we all started out once. :)

Ah, and to obsolete the ones attached falsely you can uncomment their lines  in the git bz attach or git bz edit commands. The other option is to do it here:
Details > Edit details > check obsolete
Comment 14 Arthur Bousquet 2019-05-31 20:03:46 UTC
Created attachment 90260 [details] [review]
Bug 21534: Always compute wildcards

You get no results when searching with an hyphen + with * in query string (or with preference QueryAutoTruncate) :
ie /cgi-bin/koha/opac-search.pl?q=saints-anges*

Looks like query-string by default does not compute wildcards, see analyze_wildcard in :
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html

Test plan :
1) Use Elasticsearch
2) Create a record with "saints-anges"
3) Search for "saints-anges" => you get results
4) Search for "saints-anges*" => you get results
5) Search for "saints-ang*" => you get results

Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Arthur Bousquet <arthur.bousquet@inlibro.com>


 Current status: Signed Off
Comment 15 Nick Clemens 2019-06-14 17:57:31 UTC
Created attachment 90614 [details] [review]
Bug 21534: Always compute wildcards

You get no results when searching with an hyphen + with * in query string (or with preference QueryAutoTruncate) :
ie /cgi-bin/koha/opac-search.pl?q=saints-anges*

Looks like query-string by default does not compute wildcards, see analyze_wildcard in :
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html

Test plan :
1) Use Elasticsearch
2) Create a record with "saints-anges"
3) Search for "saints-anges" => you get results
4) Search for "saints-anges*" => you get results
5) Search for "saints-ang*" => you get results

Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Arthur Bousquet <arthur.bousquet@inlibro.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 16 Martin Renvoize 2019-06-15 07:02:43 UTC
Nice work!

Pushed to master for 19.11.00
Comment 17 Fridolin SOMERS 2019-06-20 12:43:58 UTC
Pushed to 19.05.x for 19.05.01
Comment 18 Lucas Gass 2019-07-26 18:08:50 UTC
backported to 18.11.x for 18.11.09