Bug 28781 - Elasticsearch facets may be inaccurate
Summary: Elasticsearch facets may be inaccurate
Status: NEW
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching - Elasticsearch (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal
Assignee: Bugs List
QA Contact:
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-07-29 13:21 UTC by Nick Clemens (kidclamp)
Modified: 2024-02-08 00:43 UTC (History)
4 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Clemens (kidclamp) 2021-07-29 13:21:46 UTC
To recreate:
1 - Set system preferences:
    SearchEngine: Elasticsearch
    FacetMaxCount: 20
    displayFacetCount: Show
2 - Using koha testing docker default data search for '*'
3 - Note the author facets like
 Beckett, Samuel (5)
 Aristophanes (3)
 De Niese, Danielle (3)
 Euripides (3)
 Aristotle (2) 
4 - Set FacetMaxCount:400
5 - Repeat search
6 - Note facets like:
 Beckett, Samuel (5)
 Stevens, W. Richard (5)
 Plato (4)
 Aristophanes (3)
 De Niese, Danielle (3) 
7 - Some top terms are missing when FacetMaxCount is passed to ES as 'size'

From the docs:
The size parameter can be set to define how many term buckets should be returned out of the overall terms list. By default, the node coordinating the search process will request each shard to provide its own top size term buckets and once all shards respond, it will reduce the results to the final list that will then be returned to the client. This means that if the number of unique terms is greater than size, the returned list is slightly off and not accurate (it could be that the term counts are slightly off and it could even be that a term that should have been in the top size buckets was not returned).
https://www.elastic.co/guide/en/elasticsearch/reference/6.8/search-aggregations-bucket-terms-aggregation.html
Comment 1 Steve, OSLRI, USA 2024-02-08 00:43:15 UTC
Had this reported today by one of our members.  Also related to the author facet as Nick reports.  A search with the author appearing in the 5th item in results, and that same author was not shown in the facets with the max records sys pref set to 75.  This bug looks quiet.  Has there been any movement or perhaps there's a different related bug?