Bug 27515 - Elasticsearch: search for singular/plural form / stemming
Summary: Elasticsearch: search for singular/plural form / stemming
Status: NEW
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching - Elasticsearch (show other bugs)
Version: unspecified
Hardware: All All
: P5 - low enhancement
Assignee: Bugs List
QA Contact:
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-01-21 19:20 UTC by Christopher Brannon
Modified: 2024-05-06 19:01 UTC (History)
5 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Christopher Brannon 2021-01-21 19:20:02 UTC
When searching for the book "The Sawdust Dream", if you search for "sawdust dream", you get the item in the results.  When you search for "sawdust dreams", you get nothing.

It would be great if Koha would check for singular iterations of words used.
Comment 1 Barbara Johnson 2021-01-22 01:39:58 UTC
Agreed
Comment 2 Katrin Fischer 2021-01-22 09:26:51 UTC
I believe the Stemming feature with Zebra should already take care of this. Is it active in your instance and have you tried some searches?
Comment 3 Katrin Fischer 2021-01-22 09:27:45 UTC
A note here: not all languages are as lucky as English where it's about added s most of the time :)
Comment 4 Christopher Brannon 2021-01-22 15:59:38 UTC
(In reply to Katrin Fischer from comment #2)
> I believe the Stemming feature with Zebra should already take care of this.
> Is it active in your instance and have you tried some searches?

We are using ElasticSearch, not Zebra.  I have no understanding of the inner-workings of all of this, but as far as my knowledge goes, this doesn't apply.  Or am I wrong?
Comment 5 Christopher Brannon 2021-01-22 16:00:39 UTC
(In reply to Katrin Fischer from comment #3)
> A note here: not all languages are as lucky as English where it's about
> added s most of the time :)

Even so, this would be a step in the right direction.  A rule applied for specific languages that would improve searching.
Comment 6 Katrin Fischer 2021-01-22 17:24:58 UTC
Hi Christopher, as I said, I think the stemming feature already does this. Please try it!
Comment 7 Katrin Fischer 2021-01-22 17:27:22 UTC
I think it doesn't apply for Elasticsearch, but not completely sure. Changing component and adding a hint to the description.
Comment 8 David Cook 2021-01-24 23:13:45 UTC
(In reply to Katrin Fischer from comment #7)
> I think it doesn't apply for Elasticsearch, but not completely sure.
> Changing component and adding a hint to the description.

Yeah at a glance client-side stemming in Koha wasn't implemented for Elasticsearch. 

It looks like Elasticsearch can be configured to use its own stemmer token filters though.
Comment 9 Caroline Cyr La Rose 2024-05-06 19:01:56 UTC
I don't know if this can help anyone. We use snowball_french as stemmer. It works really well (with some exceptions).

https://snowballstem.org/

In index_config.yaml, under analyzer_standard, there is snowball_french under the filters

      analyzer_standard:
        tokenizer: icu_tokenizer
        filter:
          - icu_folding
          - snowball_french

and under filter (before char_filter), there is snowball_french

    filter:
      snowball_french:
        type: snowball
        language: french