Summary: | Add max_result_window to index config | ||
---|---|---|---|
Product: | Koha | Reporter: | Fridolin Somers <fridolin.somers> |
Component: | Searching - Elasticsearch | Assignee: | Fridolin Somers <fridolin.somers> |
Status: | CLOSED FIXED | QA Contact: | Victor Grousset/tuxayo <victor> |
Severity: | normal | ||
Priority: | P5 - low | CC: | arthur.suzuki, david, lucas, m.de.rooy, victor |
Version: | Main | ||
Hardware: | All | ||
OS: | All | ||
See Also: |
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27043 https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30883 |
||
Change sponsored?: | --- | Patch complexity: | Trivial patch |
Documentation contact: | Documentation submission: | ||
Text to go in the release notes: |
This updates the number of results set by default in Elasticsearch for the setting "index.max-result-window" from 10,000 to 1,000,000. This can be useful for really large catalogs.
|
Version(s) released in: |
22.11.00, 22.05.05, 21.11.12
|
Circulation function: | |||
Attachments: |
Bug 30882: Add max_result_window to index config
Bug 30882: Add max_result_window to index config Bug 30882: Add max_result_window to index config Bug 30882: (QA follow-up) Fix tests |
Description
Fridolin Somers
2022-06-02 02:20:45 UTC
This can be usefull for really big catalogs Created attachment 135572 [details] [review] Bug 30882: Add max_result_window to index config Elasticsearch number of results is by default limited by setting "index.max-result-window", default value is 10000. https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html#index-max-result-window We use this setting : https://git.koha-community.org/Koha-community/Koha/src/commit/44d6528b566e5c16eba9b15f8aa24667293766c3/Koha/SearchEngine/Elasticsearch/Search.pm#L411 I propose we add this setting in index config. Test plan : 1) Use Elasticsearch 2) Apply patch and flush memcached 3) Rebuild indexes : misc/search_tools/rebuild_elasticsearch.pl -v -b -d 4) Check the settings of index : curl '<cluster>:9200/<myindex>_biblios/_settings?pretty&filter_path=**.max_result_window' 5) You should see : "max_result_window" : "1000000" Hi Fridolin. Using koha-testing-docker, how do I get step 4 to work? The curl command provided does not work for me. Thanks! David (In reply to David Nind from comment #3) > Hi Fridolin. > > Using koha-testing-docker, how do I get step 4 to work? > > The curl command provided does not work for me. > > Thanks! > > David In KTD, use : curl 'es:9200/koha_kohadev_biblios/_settings?pretty&filter_path=**.max_result_window' (In reply to Fridolin Somers from comment #4) > In KTD, use : > curl > 'es:9200/koha_kohadev_biblios/_settings?pretty&filter_path=**. > max_result_window' Thanks Fridolin! That worked after adding this to the es section in koha-testing-docker's docker-compose.yml (after the networks configuration): ports: - "9200:9300" However, the result I got was: "max_result_window" : "10000" I think that is because this is the value in the patch (index.max_result_window: 10000)! Should that be index.max_result_window: 1000000? 8-) Ahhh good catch. Could you fix the test plan when you push your patch ? (In reply to Fridolin Somers from comment #6) > Ahhh good catch. > Could you fix the test plan when you push your patch ? No problem! Created attachment 135751 [details] [review] Bug 30882: Add max_result_window to index config Elasticsearch number of results is by default limited by setting "index.max-result-window", default value is 10000. https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html#index-max-result-window We use this setting: https://git.koha-community.org/Koha-community/Koha/src/commit/44d6528b566e5c16eba9b15f8aa24667293766c3/Koha/SearchEngine/Elasticsearch/Search.pm#L411 I propose we add this setting in index config. Test plan: 1) Use Elasticsearch 2) Apply patch and flush memcached 3) Rebuild indexes: misc/search_tools/rebuild_elasticsearch.pl -v -b -d 4) Check the settings of index (when using koha-testing-docker*): curl 'es:9200/koha_kohadev_biblios/_settings?pretty&filter_path=**.max_result_window' 5) You should see: "max_result_window" : "1000000" * You also need to add this setting to the es section in koha-testing-docker's docker-compose.yml (after the networks configuration): ports: - "9200:9300" Signed-off-by: David Nind <david@davidnind.com> I amended the patch to change 10000 to 1000000. I also amended the commit message to update the test plan - updated the curl statement and added koha-testing-docker information. (In reply to David Nind from comment #9) > I amended the patch to change 10000 to 1000000. Thats a courageous change! I agree that 10K is too low. (In reply to Marcel de Rooy from comment #10) > Thats a courageous change! Fridolin is the courageous one! Created attachment 137445 [details] [review] Bug 30882: Add max_result_window to index config Elasticsearch number of results is by default limited by setting "index.max-result-window", default value is 10000. https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html#index-max-result-window We use this setting: https://git.koha-community.org/Koha-community/Koha/src/commit/44d6528b566e5c16eba9b15f8aa24667293766c3/Koha/SearchEngine/Elasticsearch/Search.pm#L411 I propose we add this setting in index config. Test plan: 1) Use Elasticsearch 2) Apply patch and flush memcached 3) Rebuild indexes: misc/search_tools/rebuild_elasticsearch.pl -v -b -d 4) Check the settings of index (when using koha-testing-docker*): curl 'es:9200/koha_kohadev_biblios/_settings?pretty&filter_path=**.max_result_window' 5) You should see: "max_result_window" : "1000000" * You also need to add this setting to the es section in koha-testing-docker's docker-compose.yml (after the networks configuration): ports: - "9200:9300" Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> I don't know what value should we have so I'm following Fridolin and Marcel ^^ Works, qa script happy, code looks good, passing QA :) Pushed to master for 22.11. Nice work everyone, thanks! Created attachment 137834 [details] [review] Bug 30882: (QA follow-up) Fix tests Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Backported to 22.05.x for 22.05.05 thx pushed to 21.11.x for 21.11.12 Not backported to oldoldstable (21.05.x). Feel free to ask if it's needed. Nothing to document, marking resolved. |