Currently in Elasticsearch a search for 'shakes' will not match 'shakespeare' but you cna force it with 'shakes*' This patchset will obey the QueryAutoTruncate to add '*' to all search terms
Created attachment 61806 [details] [review] Bug 18374 - Respect QueryAutoTruncate syspref in Elasticsearch This patchset adds a subroutine '_truncate_terms' to the ES QueryParser. If QueryAutoTruncate is enabled this function will be called for any search to add wildcard '*' to all terms To test: 1 - Enable Elasticsearch and have some records indexed 2 - Search for partial terms 3 - Note they fail unless '*' is appended 4 - Apply patch, leave QueryAutoTruncate disabled 5 - Note partial term searches still fail 6 - Enable QueryAutoTruncate 7 - Note partial term searches succeed 8 - Do some regular and advanced searches to make sure results are as expected
Nick, please provide tests.
Created attachment 61952 [details] [review] Bug 18374 - QueryAutoTruncate unit tests To test: 1 - Apply just this patch - should fail 2 - Apply the second patch - should pass https://bugs.koha-community.org/show_bug.cgi?id=18347
Created attachment 61953 [details] [review] Bug 18374 - Respect QueryAutoTruncate syspref in Elasticsearch This patchset adds a subroutine '_truncate_terms' to the ES QueryParser. If QueryAutoTruncate is enabled this function will be called for any search to add wildcard '*' to all terms To test: 1 - Enable Elasticsearch and have some records indexed 2 - Search for partial terms 3 - Note they fail unless '*' is appended 4 - Apply patch, leave QueryAutoTruncate disabled 5 - Note partial term searches still fail 6 - Enable QueryAutoTruncate 7 - Note partial term searches succeed 8 - Do some regular and advanced searches to make sure results are as expected https://bugs.koha-community.org/show_bug.cgi?id=18347
Created attachment 61954 [details] [review] Bug 18374 - QueryAutoTruncate unit tests To test: 1 - Apply just this patch - should fail 2 - Apply the other patch - should pass https://bugs.koha-community.org/show_bug.cgi?id=18347
Created attachment 61955 [details] [review] Bug 18374 - Respect QueryAutoTruncate syspref in Elasticsearch This patchset adds a subroutine '_truncate_terms' to the ES QueryParser. If QueryAutoTruncate is enabled this function will be called for any search to add wildcard '*' to all terms To test: 1 - Enable Elasticsearch and have some records indexed 2 - Search for partial terms 3 - Note they fail unless '*' is appended 4 - Apply patch, leave QueryAutoTruncate disabled 5 - Note partial term searches still fail 6 - Enable QueryAutoTruncate 7 - Note partial term searches succeed 8 - Do some regular and advanced searches to make sure results are as expected https://bugs.koha-community.org/show_bug.cgi?id=18347
Created attachment 61983 [details] [review] Bug 18374 - Respect QueryAutoTruncate syspref in Elasticsearch This patchset adds a subroutine '_truncate_terms' to the ES QueryParser. If QueryAutoTruncate is enabled this function will be called for any search to add wildcard '*' to all terms To test: 1 - Enable Elasticsearch and have some records indexed 2 - Search for partial terms 3 - Note they fail unless '*' is appended 4 - Apply patch, leave QueryAutoTruncate disabled 5 - Note partial term searches still fail 6 - Enable QueryAutoTruncate 7 - Note partial term searches succeed 8 - Do some regular and advanced searches to make sure results are as expected https://bugs.koha-community.org/show_bug.cgi?id=18347
# Failed test at t/db_dependent/Koha_SearchEngine_Elasticsearch_Search.t line 112. # got: 'donald duck' # expected: 'donald* duck*' # Failed test at t/db_dependent/Koha_SearchEngine_Elasticsearch_Search.t line 114. # got: 'donald or duck and mickey not mouse' # expected: 'donald* or duck* and mickey* not mouse*'
Created attachment 64903 [details] [review] Bug 18374 - QueryAutoTruncate unit tests To test: 1 - Apply just this patch - should fail 2 - Apply the other patch - should pass https://bugs.koha-community.org/show_bug.cgi?id=18347
Created attachment 64904 [details] [review] Bug 18374 - Respect QueryAutoTruncate syspref in Elasticsearch This patchset adds a subroutine '_truncate_terms' to the ES QueryParser. If QueryAutoTruncate is enabled this function will be called for any search to add wildcard '*' to all terms To test: 1 - Enable Elasticsearch and have some records indexed 2 - Search for partial terms 3 - Note they fail unless '*' is appended 4 - Apply patch, leave QueryAutoTruncate disabled 5 - Note partial term searches still fail 6 - Enable QueryAutoTruncate 7 - Note partial term searches succeed 8 - Do some regular and advanced searches to make sure results are as expected https://bugs.koha-community.org/show_bug.cgi?id=18347
Created attachment 67098 [details] [review] Bug 18374: QueryAutoTruncate unit tests To test: 1 - Apply just this patch - should fail 2 - Apply the other patch - should pass Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Created attachment 67099 [details] [review] Bug 18374: Respect QueryAutoTruncate syspref in Elasticsearch This patchset adds a subroutine '_truncate_terms' to the ES QueryParser. If QueryAutoTruncate is enabled this function will be called for any search to add wildcard '*' to all terms To test: 1 - Enable Elasticsearch and have some records indexed 2 - Search for partial terms 3 - Note they fail unless '*' is appended 4 - Apply patch, leave QueryAutoTruncate disabled 5 - Note partial term searches still fail 6 - Enable QueryAutoTruncate 7 - Note partial term searches succeed 8 - Do some regular and advanced searches to make sure results are as expected Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Created attachment 67100 [details] [review] Bug 18374: (QA followup) Remove warning in Elasticsearch/Search.pm Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Created attachment 67467 [details] [review] Bug 18374: QueryAutoTruncate unit tests To test: 1 - Apply just this patch - should fail 2 - Apply the other patch - should pass Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Created attachment 67468 [details] [review] Bug 18374: Respect QueryAutoTruncate syspref in Elasticsearch This patchset adds a subroutine '_truncate_terms' to the ES QueryParser. If QueryAutoTruncate is enabled this function will be called for any search to add wildcard '*' to all terms To test: 1 - Enable Elasticsearch and have some records indexed 2 - Search for partial terms 3 - Note they fail unless '*' is appended 4 - Apply patch, leave QueryAutoTruncate disabled 5 - Note partial term searches still fail 6 - Enable QueryAutoTruncate 7 - Note partial term searches succeed 8 - Do some regular and advanced searches to make sure results are as expected Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Created attachment 67469 [details] [review] Bug 18374: (QA followup) Remove warning in Elasticsearch/Search.pm Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Created attachment 67470 [details] [review] Bug 18374: (QA follow-up) Tidy up code Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Please add more tests: - donald duck and the mouse - * - "donald duck" and "the mouse" - field:value I am sure you will see that it needs more work :)
Created attachment 67727 [details] [review] Bug 18374: Followup - Add support and tests for quoted strings To test: 1 - prove t/db_dependent/Koha_SearchEngine_Elasticsearch_Search.t 2 - do some searches in staff client and test results
(In reply to Jonathan Druart from comment #18) > Please add more tests: > - donald duck and the mouse > - * > - "donald duck" and "the mouse" > - field:value > > I am sure you will see that it needs more work :) Thanks Jonathan, followup provided, let me know if you think of more cases
Created attachment 68062 [details] [review] Bug 18374: QueryAutoTruncate unit tests To test: 1 - Apply just this patch - should fail 2 - Apply the other patch - should pass Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Created attachment 68063 [details] [review] Bug 18374: Respect QueryAutoTruncate syspref in Elasticsearch This patchset adds a subroutine '_truncate_terms' to the ES QueryParser. If QueryAutoTruncate is enabled this function will be called for any search to add wildcard '*' to all terms To test: 1 - Enable Elasticsearch and have some records indexed 2 - Search for partial terms 3 - Note they fail unless '*' is appended 4 - Apply patch, leave QueryAutoTruncate disabled 5 - Note partial term searches still fail 6 - Enable QueryAutoTruncate 7 - Note partial term searches succeed 8 - Do some regular and advanced searches to make sure results are as expected Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Created attachment 68064 [details] [review] Bug 18374: (QA followup) Remove warning in Elasticsearch/Search.pm Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Created attachment 68065 [details] [review] Bug 18374: (QA follow-up) Tidy up code Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Created attachment 68066 [details] [review] Bug 18374: Followup - Add support and tests for quoted strings To test: 1 - prove t/db_dependent/Koha_SearchEngine_Elasticsearch_Search.t 2 - do some searches in staff client and test results Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Created attachment 68067 [details] [review] Bug 18374: (QA follow-up) Simplify _truncate_terms By using a different split regex, we can simplify a bit the process of appending '*' to every word of the query Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Created attachment 68068 [details] [review] Bug 18374: (QA follow-up) Fix auto truncation for field:"value" Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Pushed to master for 17.11, thanks to everybody involved!
Pushed to 17.05.x, will be in 17.05.06.
Missing too many Elasticsearch patches in 16.11.x by now - skipping.