It is possible to extract the ids for the resultset without making ES return the records (documents) themselves. This would save processing time in both ends, and memory (building a 5000 docs resultset, serializing it, and deserializing it, needs to take resources).
Created attachment 174115 [details] [review] Bug 38386: Make compare_es_to_db.pl not retrieve the records from ES Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
in order to understand the difference in the behavior, you need an ES-enabled KTD: ```shell ktd --proxy --es7 up -d ktd --shell ``` Then compare the output of the following commands: ```shell curl -X GET 'http://es:9200/koha_kohadev_biblios/_search?scroll=1m&size=10' -H 'Content-Type: application/json' -d'{"query":{"match_all":{}}}' ``` and ```shell curl -X GET 'http://es:9200/koha_kohadev_biblios/_search?scroll=1m&size=10' -H 'Content-Type: application/json' -d'{"_source":false,"query":{"match_all":{}}}' ```
Created attachment 174583 [details] [review] Bug 38386: Make compare_es_to_db.pl not retrieve the records from ES Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Small change, big impact, passing QA