Bug 22997 - Searching gives no results in auth_finder.pl
Summary: Searching gives no results in auth_finder.pl
Status: Failed QA
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching - Elasticsearch (show other bugs)
Version: master
Hardware: All All
: P5 - low critical with 10 votes (vote)
Assignee: Fridolin SOMERS
QA Contact: Martin Renvoize
URL:
Keywords:
Depends on: 21084
Blocks:
  Show dependency treegraph
 
Reported: 2019-05-28 13:46 UTC by Fridolin SOMERS
Modified: 2019-08-11 10:06 UTC (History)
5 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Trivial patch
Who signed the patch off:
Text to go in the release notes:
Version(s) released in:


Attachments
Bug 22997: Searching gives no results in auth_finder.pl (2.24 KB, patch)
2019-05-28 14:09 UTC, Fridolin SOMERS
Details | Diff | Splinter Review
Bug 22997: Searching gives no results in auth_finder.pl (2.31 KB, patch)
2019-06-28 14:57 UTC, Kyle M Hall
Details | Diff | Splinter Review
Bug 22997: Unit text (1.24 KB, patch)
2019-08-09 11:52 UTC, Fridolin SOMERS
Details | Diff | Splinter Review
Bug 22997: Unit test (1.24 KB, patch)
2019-08-09 11:54 UTC, Fridolin SOMERS
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Fridolin SOMERS 2019-05-28 13:46:01 UTC
During cataloguing of an existing biblio, on an heading field,
the use of tag editor fills authorities finder with existing value :
  Search main heading ($a only)
  Search main heading
Default operator beeing 'contains'.

Actually with Elasticsearch those search give no results.
Comment 1 Fridolin SOMERS 2019-05-28 13:48:09 UTC
Looks like with a 'query_string' the wildcards are by default not detected.

See https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html

analyze_wildcard
	
By default, wildcards terms in a query string are not analyzed. By setting this value to true, a best effort will be made to analyze those as well.
Comment 2 Fridolin SOMERS 2019-05-28 14:09:11 UTC
Created attachment 90154 [details] [review]
Bug 22997: Searching gives no results in auth_finder.pl

During cataloguing of an existing biblio, on an heading field,
the use of tag editor fills authorities finder with existing value :
  Search main heading ($a only)
  Search main heading
Default operator beeing 'contains'.

Actually with Elasticsearch those search give no results.

Example with heading :
200
  $a Casaubon
  $b Isaac
  $f 1559-1614

Call to Elasticsearch :
    "query" : {
        "bool" : {
           "must" : [
              {
             "query_string" : {
                  "query" : "Casaubon*",
                  "default_field" : "heading-main",
                   }
              },
              {
             "query_string" : {
                  "query" : "(Isaac*) AND (1559-1614*)",
                  "default_field" : "heading"
                   }
              }
            ]
         }
      },
    "sort" : [
       {
      "heading__sort.phrase" : "asc"
       }
     ]
  }

Patch adds to "query_string" :
  analyze_wildcard : true.

Test plan :
1) Use Elasticsearch
2) Edit an existing biblio record
3) Use tag editor on a heading
4) Click search => You get correct results
5) Check also search in authorities-home.pl
Comment 3 Kyle M Hall 2019-06-28 14:57:52 UTC
Created attachment 91095 [details] [review]
Bug 22997: Searching gives no results in auth_finder.pl

During cataloguing of an existing biblio, on an heading field,
the use of tag editor fills authorities finder with existing value :
  Search main heading ($a only)
  Search main heading
Default operator beeing 'contains'.

Actually with Elasticsearch those search give no results.

Example with heading :
200
  $a Casaubon
  $b Isaac
  $f 1559-1614

Call to Elasticsearch :
    "query" : {
        "bool" : {
           "must" : [
              {
             "query_string" : {
                  "query" : "Casaubon*",
                  "default_field" : "heading-main",
                   }
              },
              {
             "query_string" : {
                  "query" : "(Isaac*) AND (1559-1614*)",
                  "default_field" : "heading"
                   }
              }
            ]
         }
      },
    "sort" : [
       {
      "heading__sort.phrase" : "asc"
       }
     ]
  }

Patch adds to "query_string" :
  analyze_wildcard : true.

Test plan :
1) Use Elasticsearch
2) Edit an existing biblio record
3) Use tag editor on a heading
4) Click search => You get correct results
5) Check also search in authorities-home.pl

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 4 Martin Renvoize 2019-07-01 15:54:01 UTC
Failing QA.. we need tests adding to Koha/SearchEngine/Elasticsearch/QueryBuilder.t for this.
Comment 5 Fridolin SOMERS 2019-08-09 11:52:32 UTC
Created attachment 92118 [details] [review]
Bug 22997: Unit text

Run prove t/db_dependent/Koha/SearchEngine/Elasticsearch/QueryBuilder.t
Comment 6 Fridolin SOMERS 2019-08-09 11:54:46 UTC
Created attachment 92119 [details] [review]
Bug 22997: Unit test

Run prove t/db_dependent/Koha/SearchEngine/Elasticsearch/QueryBuilder.t
Comment 7 Katrin Fischer 2019-08-11 09:58:39 UTC
Counting Kyle's sign-off here, switching to signed off.
Comment 8 Katrin Fischer 2019-08-11 10:06:05 UTC
I am seeing some issues here - when I used the linking on an existing sample record I got:

Search main heading ($a only), contains: Waeschke, Hermannus,
search main headin, contains: 1850- [from old catalog]


Resulting in:

[Request] ** [http://localhost:9200]-[400] [query_shard_exception] Failed to parse query [(1850-) AND ([from*) AND (old*) AND (catalog])], with: {"index":"koha_kohadev_authorities","index_uuid":"S12XDtQeTMWLoBsmUsRlnw"}, called from sub Search::Elasticsearch::Role::Client::Direct::__ANON__ at /home/vagrant/kohaclone/Koha/SearchEngine/Elasticsearch/Search.pm line 100. With vars: {'body' => {'status' => 400,'error' => {'phase' => 'query','grouped' => bless( do{\(my $o = 1)}, 'JSON::PP::Boolean' ),'root_cause' => [{'type' => 'query_shard_exception','reason' => 'Failed to parse query [(1850-) AND ([from*) AND (old*) AND (catalog])]','index_uuid' => 'S12XDtQeTMWLoBsmUsRlnw','index' => 'koha_kohadev_authorities'}],'type' => 'search_phase_execution_exception','reason' => 'all shards failed','failed_shards' => [{'index' => 'koha_kohadev_authorities','shard' => 0,'node' => 'Secs-Ua8SyyeB_6hIXEnSA','reason' => {'type' => 'query_shard_exception','reason' => 'Failed to parse query [(1850-) AND ([from*) AND (old*) AND (catalog])]','index' => 'koha_kohadev_authorities','index_uuid' => 'S12XDtQeTMWLoBsmUsRlnw','caused_by' => {'caused_by' => {'type' => 'parse_exception','reason' => 'Encountered " <RANGE_GOOP> "AND "" at line 1, column 21.
Was expecting:
    "TO" ...
    '},'reason' => 'Cannot parse \'(1850-) AND ([from*) AND (old*) AND (catalog])\': Encountered " <RANGE_GOOP> "AND "" at line 1, column 21.
Was expecting:
    "TO" ...
    ','type' => 'parse_exception'}}}]}},'request' => {'serialize' => 'std','qs' => {},'path' => '/koha_kohadev_authorities/_search','method' => 'GET','ignore' => [],'mime_type' => 'application/json','body' => {'query' => {'bool' => {'must' => [{'query_string' => {'query' => '(Waeschke,) AND (Hermannus,)','default_field' => 'heading-main','analyze_wildcard' => bless( do{\(my $o = 1)}, 'JSON::PP::Boolean' )}},{'query_string' => {'query' => '(1850-) AND ([from*) AND (old*) AND (catalog])','default_field' => 'heading','analyze_wildcard' => $VAR1->{'request'}{'body'}{'query'}{'bool'}{'must'}[0]{'query_string'}{'analyze_wildcard'}}}]}},'from' => 0,'size' => 20,'sort' => [{'heading__sort.phrase' => 'asc'}]}},'status_code' => 400}
Unable to perform your search. Please try again.

When I remove the second input, it works with and without the patch?