Bug 24555

Summary: POC Elasticsearch - Use boolean queries instead of full text queries
Product: Koha Reporter: Alex Arnaud <alex.arnaud>
Component: Searching - ElasticsearchAssignee: Alex Arnaud <alex.arnaud>
Status: Patch doesn't apply --- QA Contact:
Severity: enhancement    
Priority: P5 - low CC: anne-cecile.bories, black23, e.betemps, julian.maurice, katrin.fischer, marjorie.barry-vila, nick, severine.queune
Version: master   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Attachments: Bug 24555: Replace query string with boolean query
Bug 24555: add a button that shows ES query in search result
Bug 24555: Replace query string with boolean query
Bug 24555: add a button that shows ES query in search result
Bug 24555: Use 'cross_fields' type for multi_match queries
Bug 24555: Use 'cross_fields' type for multi_match queries
Bug 24555: Remove 'mc-' prefix in facet field name
Bug 24555: Use operator "and" for match query
Bug 24555: Fix search limits
Bug 24555: Fix search limits
Bug 24555: Fix searches like 'field-name:term'

Description Alex Arnaud 2020-01-31 16:01:33 UTC
Here is a proof of concept for replacing current use of full text queries (with query_string) by boolean queries on biblio searches.

Advantage:
    => No search craches with special character ( "!", ")" etc...)
    => Separate query context and filter context:
        - Search scores are not altered by facets,
        - Ability to use range filter  (i.e for publication date facet)

Also, this POC adds a button "ES query" in the search results header so you can see the query sent to ES.

Code is not beautiful, there is no unit test, it's a POC.
Comment 1 Alex Arnaud 2020-01-31 16:04:42 UTC
Created attachment 98242 [details] [review]
Bug 24555: Replace query string with boolean query

Test plan:
 Just make simple and advanced searches
Comment 2 Alex Arnaud 2020-01-31 16:04:49 UTC
Created attachment 98243 [details] [review]
Bug 24555: add a button that shows ES query in search result
Comment 3 Michal Denar 2020-01-31 20:32:35 UTC
Created attachment 98247 [details] [review]
Bug 24555: Replace query string with boolean query

Test plan:
 Just make simple and advanced searches

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 4 Michal Denar 2020-01-31 20:32:39 UTC
Created attachment 98248 [details] [review]
Bug 24555: add a button that shows ES query in search result

Signed-off-by: Michal Denar <black23@gmail.com>
Comment 5 Julian Maurice 2020-05-06 13:24:29 UTC
Created attachment 104419 [details] [review]
Bug 24555: Use 'cross_fields' type for multi_match queries

It allows to search in all fields as if it were one big field.

It also uses 'and' as operator, which means that all searched terms must
appear at least in one field for a document to match

https://www.elastic.co/guide/en/elasticsearch/reference/6.8/query-dsl-multi-match-query.html#type-cross-fields
Comment 6 Julian Maurice 2020-05-06 13:50:16 UTC
Created attachment 104422 [details] [review]
Bug 24555: Use 'cross_fields' type for multi_match queries

It allows to search in all fields as if it were one big field.

It also uses 'and' as operator, which means that all searched terms must
appear at least in one field for a document to match

https://www.elastic.co/guide/en/elasticsearch/reference/6.8/query-dsl-multi-match-query.html#type-cross-fields
Comment 7 Julian Maurice 2020-05-19 08:24:21 UTC
Created attachment 105061 [details] [review]
Bug 24555: Remove 'mc-' prefix in facet field name
Comment 8 Julian Maurice 2020-06-01 11:23:21 UTC
Created attachment 105470 [details] [review]
Bug 24555: Use operator "and" for match query
Comment 9 Julian Maurice 2020-06-01 11:23:55 UTC
Created attachment 105471 [details] [review]
Bug 24555: Fix search limits
Comment 10 Julian Maurice 2020-06-01 12:04:52 UTC
Created attachment 105472 [details] [review]
Bug 24555: Fix search limits
Comment 11 Julian Maurice 2020-06-11 12:37:20 UTC
Created attachment 105744 [details] [review]
Bug 24555: Fix searches like 'field-name:term'

This is used for instance in Koha::Authorities::get_usage_count
("an:<authid>")
Comment 12 Séverine Queune 2021-04-14 10:59:00 UTC
So sad :/

Applying: Bug 24555: Replace query string with boolean query
Using index info to reconstruct a base tree...
M       Koha/SearchEngine/Elasticsearch/QueryBuilder.pm
Falling back to patching base and 3-way merge...
Auto-merging Koha/SearchEngine/Elasticsearch/QueryBuilder.pm
CONFLICT (content): Merge conflict in Koha/SearchEngine/Elasticsearch/QueryBuilder.pm
error: Failed to merge in the changes.
Comment 13 Katrin Fischer 2023-12-31 13:56:32 UTC
What is the status of this?