Bug 20273 - Elasticsearch: Auth-finder.pl autocomplete must use search_auth_compat
Summary: Elasticsearch: Auth-finder.pl autocomplete must use search_auth_compat
Status: RESOLVED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching - Elasticsearch (show other bugs)
Version: master
Hardware: All All
: P5 - low normal (vote)
Assignee: Alex Arnaud
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on: 17373 19582 19604
Blocks: 21084
  Show dependency treegraph
 
Reported: 2018-02-22 14:50 UTC by Alex Arnaud
Modified: 2018-09-21 10:06 UTC (History)
7 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Bot Control: ---
When did the bot last check this:
Who signed the patch off:
Text to go in the release notes:


Attachments
Bug 20273 - Use compat routines for autocomplete in auth_finder.pl (1.92 KB, patch)
2018-02-22 15:39 UTC, Alex Arnaud
Details | Diff | Splinter Review
Bug 20273 - Use compat routines for autocomplete in auth_finder.pl (3.15 KB, patch)
2018-03-27 09:16 UTC, Alex Arnaud
Details | Diff | Splinter Review
Bug 20273 - Use compat routines for autocomplete in auth_finder.pl (3.15 KB, patch)
2018-05-03 09:42 UTC, Alex Arnaud
Details | Diff | Splinter Review
Bug 20273: Followup - Autocomplete use start instead of contains (1.65 KB, patch)
2018-05-03 09:42 UTC, Alex Arnaud
Details | Diff | Splinter Review
Bug 20273 - Use compat routines for autocomplete in auth_finder.pl (3.30 KB, patch)
2018-05-03 09:56 UTC, Séverine Queune
Details | Diff | Splinter Review
Bug 20273: Followup - Autocomplete use start instead of contains (1.80 KB, patch)
2018-05-03 09:56 UTC, Séverine Queune
Details | Diff | Splinter Review
Bug 20273 - Use compat routines for autocomplete in auth_finder.pl (3.30 KB, patch)
2018-05-03 09:57 UTC, Séverine Queune
Details | Diff | Splinter Review
Bug 20273: Followup - Autocomplete use start instead of contains (1.80 KB, patch)
2018-05-03 09:57 UTC, Séverine Queune
Details | Diff | Splinter Review
Bug 20273: Use compat routines for autocomplete in auth_finder.pl (3.30 KB, patch)
2018-07-18 19:28 UTC, Katrin Fischer
Details | Diff | Splinter Review
Bug 20273: (follow-up) Autocomplete use start instead of contains (1.80 KB, patch)
2018-07-18 19:28 UTC, Katrin Fischer
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Alex Arnaud 2018-02-22 14:50:34 UTC
BZ 19582 make auth_finder.pl use search_auth_compat instead of SearchAuthorities. It remains to do the same to the autocomplete feature provided for each search fields.
See comment: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19582#c8
Comment 1 Alex Arnaud 2018-02-22 15:39:20 UTC Comment hidden (obsolete)
Comment 2 Nicolas Legrand 2018-03-14 08:42:58 UTC Comment hidden (obsolete)
Comment 3 Nicolas Legrand 2018-03-16 11:16:40 UTC Comment hidden (obsolete)
Comment 4 Alex Arnaud 2018-03-27 09:16:49 UTC
Created attachment 73307 [details] [review]
Bug 20273 - Use compat routines for autocomplete in auth_finder.pl

Test plan:
  - Context: Koha working with elasticsearch,
  - Apply BZ 19582 (that make auth_finder.pl works again),
  - apply this patch,
  - go to cgi-bin/koha/authorities/auth_finder.pl,
  - type at least 3 chararcters in a search field and
    check that the autocomplete returns results
Comment 5 Nicolas Legrand 2018-03-28 10:10:35 UTC
The ysearch.pl use 'contains' operator instead of 'start'. When adjusting the search form to 'start with', ysearch.pl still launch a 'contains' operator. It means we need to put a complete word to make it work.

Is it the wanted behaviour? Wouldn't it be better to put a start operator by default? When I type three letters and see the wheel spinning without results, it feels like nothing looks like what I typed.
Comment 6 Alex Arnaud 2018-04-04 15:56:13 UTC
(In reply to Nicolas Legrand from comment #5)
> The ysearch.pl use 'contains' operator instead of 'start'. When adjusting
> the search form to 'start with', ysearch.pl still launch a 'contains'
> operator. It means we need to put a complete word to make it work.
> 
> Is it the wanted behaviour? Wouldn't it be better to put a start operator by
> default? When I type three letters and see the wheel spinning without
> results, it feels like nothing looks like what I typed.

I'm not sure of the right answer, but i'm not upset by that. Operators are used for the search itself, not for autocomplete.
Comment 7 Alex Arnaud 2018-05-03 09:42:17 UTC
Created attachment 74996 [details] [review]
Bug 20273 - Use compat routines for autocomplete in auth_finder.pl

Test plan:
  - Context: Koha working with elasticsearch,
  - Apply BZ 19582 (that make auth_finder.pl works again),
  - apply this patch,
  - go to cgi-bin/koha/authorities/auth_finder.pl,
  - type at least 3 chararcters in a search field and
    check that the autocomplete returns results
Comment 8 Alex Arnaud 2018-05-03 09:42:27 UTC
Created attachment 74997 [details] [review]
Bug 20273: Followup - Autocomplete use start instead of contains
Comment 9 Séverine Queune 2018-05-03 09:56:53 UTC
Created attachment 74998 [details] [review]
Bug 20273 - Use compat routines for autocomplete in auth_finder.pl

Test plan:
  - Context: Koha working with elasticsearch,
  - Apply BZ 19582 (that make auth_finder.pl works again),
  - apply this patch,
  - go to cgi-bin/koha/authorities/auth_finder.pl,
  - type at least 3 chararcters in a search field and
    check that the autocomplete returns results

Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>
Comment 10 Séverine Queune 2018-05-03 09:56:56 UTC
Created attachment 74999 [details] [review]
Bug 20273: Followup - Autocomplete use start instead of contains

Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>
Comment 11 Séverine Queune 2018-05-03 09:57:30 UTC
Created attachment 75000 [details] [review]
Bug 20273 - Use compat routines for autocomplete in auth_finder.pl

Test plan:
  - Context: Koha working with elasticsearch,
  - Apply BZ 19582 (that make auth_finder.pl works again),
  - apply this patch,
  - go to cgi-bin/koha/authorities/auth_finder.pl,
  - type at least 3 chararcters in a search field and
    check that the autocomplete returns results

Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>
Comment 12 Séverine Queune 2018-05-03 09:57:34 UTC
Created attachment 75001 [details] [review]
Bug 20273: Followup - Autocomplete use start instead of contains

Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>
Comment 13 Katrin Fischer 2018-07-08 13:02:12 UTC
I'd love to test this, but I am stuck on my Elasticsearch authority search not working at all. 

This is what I've tried:

I've reset my mappings using:
http://localhost:8081/cgi-bin/koha/admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1

Then I've done a complete reindex.

When I search for an authority, I get an Internal server error resolving to:

[Request] ** [http://localhost:9200]-[400] [query_shard_exception] No mapping found for [Heading__sort.phrase] in order to sort on, with: {"index":"koha_kohadev_authorities","index_uuid":"FwdgBXbyRq-5E0F3sa4LTA"}, called from sub Search::Elasticsearch::Role::Client::Direct::__ANON__ at /usr/share/perl5/Catmandu/Store/ElasticSearch/Bag.pm line 167. With vars: {'request' => {'method' => 'GET','path' => '/koha_kohadev_authorities/data/_search','ignore' => [],'mime_type' => 'application/json','qs' => {},'body' => {'sort' => [{'Heading__sort.phrase' => 'asc'}],'query' => {'bool' => {'should' => [{'match' => {'Heading-Main' => 'k'}}]}},'size' => 20,'from' => 0},'serialize' => 'std'},'status_code' => 400,'body' => {'status' => 400,'error' => {'type' => 'search_phase_execution_exception','grouped' => bless( do{\(my $o = 1)}, 'JSON::PP::Boolean' ),'failed_shards' => [{'shard' => 0,'reason' => {'index' => 'koha_kohadev_authorities','type' => 'query_shard_exception','reason' => 'No mapping found for [Heading__sort.phrase] in order to sort on','index_uuid' => 'FwdgBXbyRq-5E0F3sa4LTA'},'node' => 'vSTPqetISteesIkQSA_Ehg','index' => 'koha_kohadev_authorities'}],'phase' => 'query','reason' => 'all shards failed','root_cause' => [{'type' => 'query_shard_exception','index' => 'koha_kohadev_authorities','reason' => 'No mapping found for [Heading__sort.phrase] in order to sort on','index_uuid' => 'FwdgBXbyRq-5E0F3sa4LTA'}]}}}
Unable to perform your search. Please try again.

Anyone able to help or point me in the right direction?
Comment 14 Nicolas Legrand 2018-07-09 11:03:57 UTC
Have you tried sort 'None' ?
Comment 15 Katrin Fischer 2018-07-09 11:51:08 UTC
I was doing a 'standard' search in authorities search, without changing any settings. What do you mean by none?
Comment 16 Nicolas Legrand 2018-07-09 14:01:06 UTC
Ther(In reply to Katrin Fischer from comment #15)
> I was doing a 'standard' search in authorities search, without changing any
> settings. What do you mean by none?

there is a sort menu after the input field which can be set to 'Heading A-Z', 'Heading Z-A' or 'None'. I remember Nick once told me it wasn't working well. Plus there is some 'sort' trouble in your log so I thought about this, but maybe it is not relevant.
Comment 17 Katrin Fischer 2018-07-09 18:45:54 UTC
Hi Nicolas, thx! 'None' works. 

Is there a bug already about the broken sorting for authorities with Elasticsearch?
Comment 18 Nicolas Legrand 2018-07-10 08:42:32 UTC
(In reply to Katrin Fischer from comment #17)
> Hi Nicolas, thx! 'None' works. 

Great!

> Is there a bug already about the broken sorting for authorities with
> Elasticsearch?

I think it's Bug 19604. I cc Nick so he can confirm.
Comment 19 Nick Clemens 2018-07-10 10:34:09 UTC
(In reply to Nicolas Legrand from comment #18)
> (In reply to Katrin Fischer from comment #17)
> > Hi Nicolas, thx! 'None' works. 
> 
> Great!
> 
> > Is there a bug already about the broken sorting for authorities with
> > Elasticsearch?
> 
> I think it's Bug 19604. I cc Nick so he can confirm.

I believe that is correct, 19604 will at least improve the sorting situation
Comment 20 Katrin Fischer 2018-07-15 15:41:10 UTC
I feel this is not an enh as the autocomplate displays, but doesn't work without this patch. Changing to 'normal'.
Comment 21 Katrin Fischer 2018-07-15 15:43:20 UTC
Sorry, but I can' test this - I switched the sort to none, but the auto-complete feature triggers the same error for the sorting. I think we need to fix this first.

[Request] ** [http://localhost:9200]-[400] [query_shard_exception] No mapping found for [Heading__sort.phrase] in order to sort on, with: {"index_uuid":"FwdgBXbyRq-5E0F3sa4LTA","index":"koha_kohadev_authorities"}, called from sub Search::Elasticsearch::Role::Client::Direct::__ANON__ at /usr/share/perl5/Catmandu/Store/ElasticSearch/Bag.pm line 167. With vars: {'body' => {'status' => 400,'error' => {'failed_shards' => [{'index' => 'koha_kohadev_authorities','node' => 'vSTPqetISteesIkQSA_Ehg','reason' => {'index' => 'koha_kohadev_authorities','index_uuid' => 'FwdgBXbyRq-5E0F3sa4LTA','reason' => 'No mapping found for [Heading__sort.phrase] in order to sort on','type' => 'query_shard_exception'},'shard' => 0}],'type' => 'search_phase_execution_exception','root_cause' => [{'type' => 'query_shard_exception','reason' => 'No mapping found for [Heading__sort.phrase] in order to sort on','index_uuid' => 'FwdgBXbyRq-5E0F3sa4LTA','index' => 'koha_kohadev_authorities'}],'reason' => 'all shards failed','phase' => 'query','grouped' => bless( do{\(my $o = 1)}, 'JSON::PP::Boolean' )}},'request' => {'path' => '/koha_kohadev_authorities/data/_search','qs' => {},'ignore' => [],'mime_type' => 'application/json','serialize' => 'std','body' => {'sort' => [{'Heading__sort.phrase' => 'asc'}],'size' => 50,'from' => 0,'query' => {'bool' => {'should' => [{'wildcard' => {'Heading-Main.phrase' => 'sew*'}}]}}},'method' => 'GET'},'status_code' => 400}
Unable to perform your search. Please try again.
Comment 22 Katrin Fischer 2018-07-15 15:48:06 UTC
It seems it can't work right now without fixing authority sorting. We could also mark it BLOCKED instead if that is preferred. Hopefully it can be resolved soon!
Comment 23 Katrin Fischer 2018-07-18 19:28:25 UTC
Created attachment 77090 [details] [review]
Bug 20273: Use compat routines for autocomplete in auth_finder.pl

Test plan:
  - Context: Koha working with elasticsearch,
  - Apply BZ 19582 (that make auth_finder.pl works again),
  - apply this patch,
  - go to cgi-bin/koha/authorities/auth_finder.pl,
  - type at least 3 chararcters in a search field and
    check that the autocomplete returns results

Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>
Comment 24 Katrin Fischer 2018-07-18 19:28:29 UTC
Created attachment 77091 [details] [review]
Bug 20273: (follow-up) Autocomplete use start instead of contains

Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>
Comment 25 Nick Clemens 2018-07-19 17:36:17 UTC
Awesome work all!

Pushed to master for 18.11
Comment 26 Martin Renvoize 2018-08-02 14:14:47 UTC
Depends on bug 19604 which needs clarifications and help in backporting to 18.05.x branch.. awaiting feedback there.
Comment 27 Martin Renvoize 2018-08-06 13:01:55 UTC
Pushed to 18.05.x for 18.05.03.
Comment 28 Fridolin SOMERS 2018-09-21 10:06:25 UTC
Depends on bugs not in 17.11.x