Current setting in elasticsearch mapping use 'text' type for sort fields, this seems like extra overhead. If the fields are only used for sorting we can index them as 'keyword' (exact phrase) and this has the side effect of fixing sorting
Created attachment 60780 [details] [review] Bug 18189 - Elasticsearch sorting broken To test: 1 - Have ES enabled and biblios and authorities indexed 2 - Perform a catalog search 3 - Try to sort results by author, it will fail 4 - Perform an authority search with sorting enabled, it will fail 5 - Apply patch 6 - Reindex, deleting current indexes perl /home/vagrant/kohaclone/misc/search_tools/rebuild_elastic_search.pl -d -v 7 - Repeat 3&4, they should succeed
Note this won't fix all sorting options unless they are defined as sortable in the config. acqdate fails out of the box - set acqdate as sortable - reindex - it should now work I suppose we either don't allow altering sortability for some fields, or the dropdown must populate based on what is defined as sortable.
I'm getting: Fielddata is disabled on text fields by default. Set fielddata=true on [author.phrase] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Is something wrong with my es config?
(In reply to Srdjan Jankovic from comment #3) > I'm getting: > Fielddata is disabled on text fields by default. Set fielddata=true on > [author.phrase] in order to load fielddata in memory by uninverting the > inverted index. Note that this can however use significant memory. > > Is something wrong with my es config? Is this before or after the patch? I cannot seem to replicate the issue.
(In reply to Nick Clemens from comment #4) > (In reply to Srdjan Jankovic from comment #3) > > I'm getting: > > Fielddata is disabled on text fields by default. Set fielddata=true on > > [author.phrase] in order to load fielddata in memory by uninverting the > > inverted index. Note that this can however use significant memory. > > > > Is something wrong with my es config? > > Is this before or after the patch? I cannot seem to replicate the issue. Both. What was the error that you were trying to get rid of?
Before this patch passing sort options to Elastic gives an error much like the one you mentioned. If you are getting this before and after the patch I would suspect it is an issue with modules/versions, and or outdated mappings. Using admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1 should get your mappings up to date, then delete and rebuild indexes, and you should be good I would also check versions against kohadevbox elastic setup: https://github.com/digibib/kohadevbox/blob/master/roles/kohadevbox/tasks/elasticsearch-2.x.yml
Hm, done the reset thing, and now getting: No mapping found for [author__sort.phrase] in order to sort on, with: {"index":"koha_biblios","index_uuid":"fehRdZHCRlGe1FxHYiKbMA"}
Created attachment 61494 [details] [review] [SIGNED-OFF] Bug 18189 - Elasticsearch sorting broken To test: 1 - Have ES enabled and biblios and authorities indexed 2 - Perform a catalog search 3 - Try to sort results by author, it will fail 4 - Perform an authority search with sorting enabled, it will fail 5 - Apply patch 6 - Reindex, deleting current indexes perl /home/vagrant/kohaclone/misc/search_tools/rebuild_elastic_search.pl -d -v 7 - Repeat 3&4, they should succeed Signed-off-by: Srdjan <srdjan@catalyst.net.nz>
Sorry, my fault, omitted -d when reindexing.
What about integers and dates?
Does only work with author, our other fields are not marked as sortable by default (?!?) Moreover "Ó Cadhain, Máirtín" is sorted after "Yerby, Frank Garvin", terrible...
Created attachment 61689 [details] [review] Bug 18189 - Elasticsearch sorting broken To test: 1 - Have ES enabled and biblios and authorities indexed 2 - Perform a catalog search 3 - Try to sort results by author, it will fail 4 - Perform an authority search with sorting enabled, it will fail 5 - Apply patch 6 - Reindex, deleting current indexes perl /home/vagrant/kohaclone/misc/search_tools/rebuild_elastic_search.pl -d -v 7 - Repeat 3&4, they should succeed Signed-off-by: Srdjan <srdjan@catalyst.net.nz> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Pushed to master for 17.05, thanks Nick!
Is enh correct here or should this not be a bug?
Sort is broken, it's a bug.
Ok, missed the dependency on bug 17255 earlier - should we still push this to stable?
3 months - still no answer here. Closing.