Discovered this issue while testing the search links in biblio-details. The links are hard-coded in the template with queries not using quotes for field values, which makes QueryBuilder::_truncate_terms append '*' to the search term. Since wild card searches skips the analyzer-phase, most of our links does not work (since a lot of field values has word boundaries and/or punctuation marks, for example the query "Provider:Taschen," will be converted to "Provider:Taschen,*" and since the indexed value for Provider is "Taschen" this search will produce no results. In my opinion (if I'm not misunderstanding the purpose of this code) appending '*' to words should be removed.
Created attachment 70227 [details] [review] Bug 19905 - Don't append wildcard to query terms Appending '*' to query terms makes Elasticsearch skip search analysis. This will for example cause issues with search links in biblio details for values other than a just single term, since the indexed value has been tokenized. How to test: 1) Find a biblio with a field value containing word boundaries and/or punctuation marks. For example: "au:Author, Name". 2) Perform a search using this query, this should not produce any results. 3) Apply patch. 4) Search using the same query again, it should now produce the expected results.
I just now discovered that the link I was testing this with (Publisher, using the query Provider:<value>, is not quoting the value, but most (perhaps all) of the other links are. So an alternative solution to this issue could be add quotes in the xsl-template. Still we have received serveral complaints from libraries failing to perform searches using unquoted field values, and though we could tell everyone to always use quotes, it feels like Koha would provide a better user experience if wildcard queries where not the default.
*librarians, not libraries.
Looking through the tests I now saw there is an "QueryAutoTruncate" option that was enabled in my instance. So turning this off will most likely fix this issue for us, and I now realize it cannot just be removed (without also removing this option), making this fix an incorrect one. I have not tested it out myself, but I think there may be other issues with the current autotruncation, should not the whole phrase be searched (the "raw" field for example) or else you will get strange results for wildcard queries containing multiple tokens (since it is the tokanized values that will be matched against)? But that is another possible issue, and should perhaps not be addressed here. But for now I'm disabling the option, and this issues should perhaps be closed. There is still an issue with the Provider-link not being quoted (and might not work if "QueryAutoTruncate" is enabled), but better to open another issue for that one.
Hi David, I see this is marked in discussion - something we can do about it?
Thanks for the comment. No, not really, this issue can be closed. I do perhaps think it's good to be aware of the fact that analysis phase of ElalistSearch is skipped for wildcard-query, so the QueryAutoTruncate-option is a little bit dangerous in that regard, and should perhaps be recommended against. But better to do something about that in another issue in that case.
Hi David, sorry, a bit lost in the discussion, please open a new report for the QueryAutoTruncate issue if you think it makes sense. Closing this one here.