Summary: | Wrong unicode tokenization | ||
---|---|---|---|
Product: | Koha | Reporter: | Nick Clemens (kidclamp) <nick> |
Component: | Searching - Elasticsearch | Assignee: | Nick Clemens (kidclamp) <nick> |
Status: | CLOSED FIXED | QA Contact: | Julian Maurice <julian.maurice> |
Severity: | major | ||
Priority: | P5 - low | CC: | claire_gravely, fridolin.somers, jonathan.druart, julian.maurice, philippe.blouin, severine.queune, tomascohen |
Version: | Main | ||
Hardware: | All | ||
OS: | All | ||
Change sponsored?: | --- | Patch complexity: | --- |
Documentation contact: | Documentation submission: | ||
Text to go in the release notes: | Version(s) released in: | ||
Circulation function: | |||
Attachments: |
Bug 18318 - Unicode support for Elasticsearch
Bug 18318: Unicode support for Elasticsearch Bug 18318: Unicode support for Elasticsearch |
Description
Nick Clemens (kidclamp)
2017-03-23 08:48:08 UTC
Created attachment 61982 [details] [review] Bug 18318 - Unicode support for Elasticsearch You must install the icu plugin for elasticsearch https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-icu.html Once installed, apply this patch Reindex your data, deleting the existing indexes perl /home/vagrant/kohaclone/misc/search_tools/rebuild_elastic_search.pl -d Find (or add) some titles with accented characters Verify that a search for the exact character or the unaccented version works Just a note, the ICU analysis plugin has been incorporated into KohaDevBox so in order to test this patch you can provision a new box as usual: KOHA_ELASTICSEARCH=1 vagrant up Regards Created attachment 67344 [details] [review] Bug 18318: Unicode support for Elasticsearch You must install the icu plugin for elasticsearch https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-icu.html Once installed, apply this patch Reindex your data, deleting the existing indexes perl /home/vagrant/kohaclone/misc/search_tools/rebuild_elastic_search.pl -d Find (or add) some titles with accented characters Verify that a search for the exact character or the unaccented version works Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> The patch does what it says, but it adds a dependency (ICU plugin) without mentioning it anywhere (AFAIK). I don't know what is the best place for this (https://wiki.koha-community.org/wiki/Elasticsearch maybe ?), but it must be written somewhere. Marking as Failed QA for now. I will pass QA as soon as there is some kind of documentation for this. (In reply to Julian Maurice from comment #4) > The patch does what it says, but it adds a dependency (ICU plugin) without > mentioning it anywhere (AFAIK). > I don't know what is the best place for this > (https://wiki.koha-community.org/wiki/Elasticsearch maybe ?), but it must be > written somewhere. > > Marking as Failed QA for now. I will pass QA as soon as there is some kind > of documentation for this. The commit message mentions itbut definitely worth documenting in a more publicly available place. I've adjusted the docs (https://wiki.koha-community.org/wiki/Elasticsearch). (In reply to Tomás Cohen Arazi from comment #5) > I've adjusted the docs (https://wiki.koha-community.org/wiki/Elasticsearch). Thanks! Created attachment 67449 [details] [review] Bug 18318: Unicode support for Elasticsearch You must install the icu plugin for elasticsearch https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-icu.html Once installed, apply this patch Reindex your data, deleting the existing indexes perl /home/vagrant/kohaclone/misc/search_tools/rebuild_elastic_search.pl -d Find (or add) some titles with accented characters Verify that a search for the exact character or the unaccented version works Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Julian Maurice <julian.maurice@biblibre.com> Pushed to master for 17.11, thanks to everybody involved! ICU support, awesome :D, thanks all. Pushed to 17.05.x, will be in 17.05.05. Patch does not apply cleanly. Closing as 16.11.x is missing quite some patches for Elasticsearch now. |