We need to offer unicode support in Elasticsearch https://www.elastic.co/guide/en/elasticsearch/guide/current/case-folding.html https://www.elastic.co/guide/en/elasticsearch/guide/current/icu-plugin.html Currently a search for: pokemon Will not return results for: pokémon That is just an example :-)
Created attachment 61982 [details] [review] Bug 18318 - Unicode support for Elasticsearch You must install the icu plugin for elasticsearch https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-icu.html Once installed, apply this patch Reindex your data, deleting the existing indexes perl /home/vagrant/kohaclone/misc/search_tools/rebuild_elastic_search.pl -d Find (or add) some titles with accented characters Verify that a search for the exact character or the unaccented version works
Just a note, the ICU analysis plugin has been incorporated into KohaDevBox so in order to test this patch you can provision a new box as usual: KOHA_ELASTICSEARCH=1 vagrant up Regards
Created attachment 67344 [details] [review] Bug 18318: Unicode support for Elasticsearch You must install the icu plugin for elasticsearch https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-icu.html Once installed, apply this patch Reindex your data, deleting the existing indexes perl /home/vagrant/kohaclone/misc/search_tools/rebuild_elastic_search.pl -d Find (or add) some titles with accented characters Verify that a search for the exact character or the unaccented version works Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
The patch does what it says, but it adds a dependency (ICU plugin) without mentioning it anywhere (AFAIK). I don't know what is the best place for this (https://wiki.koha-community.org/wiki/Elasticsearch maybe ?), but it must be written somewhere. Marking as Failed QA for now. I will pass QA as soon as there is some kind of documentation for this.
(In reply to Julian Maurice from comment #4) > The patch does what it says, but it adds a dependency (ICU plugin) without > mentioning it anywhere (AFAIK). > I don't know what is the best place for this > (https://wiki.koha-community.org/wiki/Elasticsearch maybe ?), but it must be > written somewhere. > > Marking as Failed QA for now. I will pass QA as soon as there is some kind > of documentation for this. The commit message mentions itbut definitely worth documenting in a more publicly available place. I've adjusted the docs (https://wiki.koha-community.org/wiki/Elasticsearch).
(In reply to Tomás Cohen Arazi from comment #5) > I've adjusted the docs (https://wiki.koha-community.org/wiki/Elasticsearch). Thanks!
Created attachment 67449 [details] [review] Bug 18318: Unicode support for Elasticsearch You must install the icu plugin for elasticsearch https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-icu.html Once installed, apply this patch Reindex your data, deleting the existing indexes perl /home/vagrant/kohaclone/misc/search_tools/rebuild_elastic_search.pl -d Find (or add) some titles with accented characters Verify that a search for the exact character or the unaccented version works Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Pushed to master for 17.11, thanks to everybody involved!
ICU support, awesome :D, thanks all.
Pushed to 17.05.x, will be in 17.05.05.
Patch does not apply cleanly. Closing as 16.11.x is missing quite some patches for Elasticsearch now.