To reproduce: 1. Set ElasticsearchMARCFormat="Searchable array" 2. Reindex all the things: $ ktd --shell k$ koha-elasticsearch --rebuild -d -v kohadev 3. Go to advanced search, using the 'More options' option. 4. Check 'Search entire MARC record' and 'Apply weights to search' 5. Perform a search on 'FULL TEXT' => SUCCESS: The URL contains `whole_record=on` and `weighted_fields=on` 6. Hover on any facet and look at the link => FAIL: It doesn't contain them!
Created attachment 187529 [details] [review] Bug 40966: Make sure 'whole_record' and 'weighted_fields' are passed around The ES query builder correctly handles this parameters (passed from search.pl and opac-search.pl) when it builds the ES query. But the returned 'query_cgi' that is used later for keeping the search results query context for building things like facets, just doesn't pass this values around, leading to a loss of them on the generated links. This patch makes sure they are passed around. To test: 1. Set ElasticsearchMARCFormat="Searchable array" 2. Reindex all the things: $ ktd --shell k$ koha-elasticsearch --rebuild -d -v kohadev 3. Go to advanced search, using the 'More options' option. 4. Check 'Search entire MARC record' and 'Apply weights to search' 5. Perform a search on 'FULL TEXT' => SUCCESS: The URL contains `whole_record=on` and `weighted_fields=on` 6. Hover on any facet and look at the link => FAIL: It doesn't contain them! 7. Apply this patch 8. Restart all k$ restart_all 9. Refresh the search results 10. Repeat 6 => SUCCESS: Links have the needed parameters! 11. Sign off :-D
Created attachment 187531 [details] [review] Bug 40966: Regression tests Tests verify that whole_record and weighted_fields parameters are correctly included in the query_cgi string returned by build_query_compat(), ensuring these parameters are preserved in facet links and pagination. To test: 1. Apply this patch 2. Run: $ ktd --shell k$ prove t/db_dependent/Koha/SearchEngine/Elasticsearch/QueryBuilder.t => FAIL: The values are not being passed around
Created attachment 187532 [details] [review] Bug 40966: Make sure 'whole_record' and 'weighted_fields' are passed around The ES query builder correctly handles this parameters (passed from search.pl and opac-search.pl) when it builds the ES query. But the returned 'query_cgi' that is used later for keeping the search results query context for building things like facets, just doesn't pass this values around, leading to a loss of them on the generated links. This patch makes sure they are passed around. To test: 1. Apply the regression tests 2. Run: $ ktd --shell k$ prove t/db_dependent/Koha/SearchEngine/Elasticsearch/QueryBuilder.t => FAIL: Tests fail! These params are not passed around! 3. Set ElasticsearchMARCFormat="Searchable array" 4. Reindex all the things: k$ koha-elasticsearch --rebuild -d -v kohadev 5. Go to advanced search, using the 'More options' option. 6. Check 'Search entire MARC record' and 'Apply weights to search' 7. Perform a search on 'FULL TEXT' => SUCCESS: The URL contains `whole_record=on` and `weighted_fields=on` 8. Hover on any facet and look at the link => FAIL: It doesn't contain them! 9. Apply this patch 10. Restart all k$ restart_all 11. Refresh the search results 12. Repeat 6 => SUCCESS: Links have the needed parameters! 13. Repeat 2 => SUCCESS: Tests pass! the parameters are passed around! 14. Sign off :-D
This fixes it for the 'on' case - but if I uncheck "Apply field weights to search" - do a search, then click a facet, the weights are applied again The issue is from: my $weight_search = $params->{'advsearch'} ? $params->{'weight_search'} || 0 : 1;
Created attachment 187945 [details] [review] Bug 40966: Regression tests Tests verify that whole_record and weighted_fields parameters are correctly included in the query_cgi string returned by build_query_compat(), ensuring these parameters are preserved in facet links and pagination. To test: 1. Apply this patch 2. Run: $ ktd --shell k$ prove t/db_dependent/Koha/SearchEngine/Elasticsearch/QueryBuilder.t => FAIL: The values are not being passed around
Created attachment 187946 [details] [review] Bug 40966: Make sure 'whole_record' and 'weighted_fields' are passed around The ES query builder correctly handles this parameters (passed from search.pl and opac-search.pl) when it builds the ES query. But the returned 'query_cgi' that is used later for keeping the search results query context for building things like facets, just doesn't pass this values around, leading to a loss of them on the generated links. This patch makes sure they are passed around. To test: 1. Apply the regression tests 2. Run: $ ktd --shell k$ prove t/db_dependent/Koha/SearchEngine/Elasticsearch/QueryBuilder.t => FAIL: Tests fail! These params are not passed around! 3. Set ElasticsearchMARCFormat="Searchable array" 4. Reindex all the things: k$ koha-elasticsearch --rebuild -d -v kohadev 5. Go to advanced search, using the 'More options' option. 6. Check 'Search entire MARC record' and 'Apply weights to search' 7. Perform a search on 'FULL TEXT' => SUCCESS: The URL contains `whole_record=on` and `weighted_fields=on` 8. Hover on any facet and look at the link => FAIL: It doesn't contain them! 9. Apply this patch 10. Restart all k$ restart_all 11. Refresh the search results 12. Repeat 6 => SUCCESS: Links have the needed parameters! 13. Repeat 2 => SUCCESS: Tests pass! the parameters are passed around! 14. Sign off :-D
For step 7, I'm assuming you put 'FULL TEXT' in the first keyword field. I'm getting this for search results, instead of results and facets (normal advanced searches work fine): No results found You did not specify any search criteria. Error: Unable to perform your search. Please try again.
(In reply to David Nind from comment #7) > For step 7, I'm assuming you put 'FULL TEXT' in the first keyword field. > > I'm getting this for search results, instead of results and facets (normal > advanced searches work fine): > > No results found > You did not specify any search criteria. > > Error: Unable to perform your search. Please try again. I chose phrase search instead of keyword.
(In reply to Tomás Cohen Arazi (tcohen) from comment #8) > (In reply to David Nind from comment #7) > > For step 7, I'm assuming you put 'FULL TEXT' in the first keyword field. > > > > I'm getting this for search results, instead of results and facets (normal > > advanced searches work fine): > > > > No results found > > You did not specify any search criteria. > > > > Error: Unable to perform your search. Please try again. > > I chose phrase search instead of keyword. I chose 'Keyword as phrase' - without 'Search entire MARC record' I get a normal no results page, with it ticked I get the same error as above. With both patches applied, the tests now pass but I get the same error. (I'm using ktd --es7 up, as I haven't been able to get es8 to work for a while on KTD.)
I had another go at testing (using ES7), but couldn't get any search results (got an error) when checking "Search entire MARC record". Testing notes, from steps 5-7: 1. Search > Advanced search 2. More options 3. Changed keyword for the first 'Search for' to 'Keyword as phrase' 4. Search term entered: agrovoc 5. Checked "Search entire MARC record" ("Apply field weights to search" was already checked") 5. I get: No results found You did not specify any search criteria. Error: Unable to perform your search. Please try again. 6. The URL includes `whole_record=on` and `weight_search=on`, but not `weighted_fields=on`. The full URL is: /cgi-bin/koha/catalogue/search.pl?advsearch=1&idx=kw%2Cphr&q=agrovoc&whole_record=on&weight_search=on&sort_by=relevance 7. Note: I do get search results when only "Apply field weights to search" s checked, with 'Subject as phrase' and 'Keyword as phrase'. I get the error if I have "Search entire MARC record" checked as well. 8. If I apply the second patch and repeat the above steps, I still get the same error as in step 5, but the URL is now: /cgi-bin/koha/catalogue/search.pl?advsearch=1&idx=kw%2Cphr&q=agrovoc&whole_record=on&weight_search_submitted=1&weight_search=on&sort_by=relevance 9. With the second patch, the tests pass. 10. All of the above was in the staff interface. For the OPAC, I don't have a "Search entire MARC record" option to check for the advanced search.
Created attachment 190740 [details] [review] Bug 40966: Regression tests Tests verify that whole_record and weighted_fields parameters are correctly included in the query_cgi string returned by build_query_compat(), ensuring these parameters are preserved in facet links and pagination. To test: 1. Apply this patch 2. Run: $ ktd --shell k$ prove t/db_dependent/Koha/SearchEngine/Elasticsearch/QueryBuilder.t => FAIL: The values are not being passed around
Created attachment 190741 [details] [review] Bug 40966: Make sure 'whole_record' and 'weighted_fields' are passed around The ES query builder correctly handles this parameters (passed from search.pl and opac-search.pl) when it builds the ES query. But the returned 'query_cgi' that is used later for keeping the search results query context for building things like facets, just doesn't pass this values around, leading to a loss of them on the generated links. This patch makes sure they are passed around. To test: 1. Apply the regression tests 2. Run: $ ktd --shell k$ prove t/db_dependent/Koha/SearchEngine/Elasticsearch/QueryBuilder.t => FAIL: Tests fail! These params are not passed around! 3. Set ElasticsearchMARCFormat="Searchable array" 4. Reindex all the things: k$ koha-elasticsearch --rebuild -d -v kohadev 5. Go to advanced search, using the 'More options' option. 6. Check 'Search entire MARC record' and 'Apply weights to search' 7. Perform a search on 'FULL TEXT' => SUCCESS: The URL contains `whole_record=on` and `weighted_fields=on` 8. Hover on any facet and look at the link => FAIL: It doesn't contain them! 9. Apply this patch 10. Restart all k$ restart_all 11. Refresh the search results 12. Repeat 6 => SUCCESS: Links have the needed parameters! 13. Repeat 2 => SUCCESS: Tests pass! the parameters are passed around! 14. Sign off :-D
Created attachment 190742 [details] [review] Bug 40966: Make sure 'whole_record' and 'weighted_fields' are passed around The ES query builder correctly handles this parameters (passed from search.pl and opac-search.pl) when it builds the ES query. But the returned 'query_cgi' that is used later for keeping the search results query context for building things like facets, just doesn't pass this values around, leading to a loss of them on the generated links. This patch makes sure they are passed around. To test: 1. Apply the regression tests 2. Run: $ ktd --shell k$ prove t/db_dependent/Koha/SearchEngine/Elasticsearch/QueryBuilder.t => FAIL: Tests fail! These params are not passed around! 3. Set ElasticsearchMARCFormat="Searchable array" 4. Reindex all the things: k$ koha-elasticsearch --rebuild -d -v kohadev 5. Go to advanced search, using the 'More options' option. 6. Check 'Search entire MARC record' and 'Apply weights to search' 7. Perform a search on 'FULL TEXT' => SUCCESS: The URL contains `whole_record=on` and `weighted_fields=on` 8. Hover on any facet and look at the link => FAIL: It doesn't contain them! 9. Apply this patch 10. Restart all k$ restart_all 11. Refresh the search results 12. Repeat 6 => SUCCESS: Links have the needed parameters! 13. Repeat 2 => SUCCESS: Tests pass! the parameters are passed around! 14. Sign off :-D
Hi, David. Thanks for taking the time! I've just started a fresh KTD like this [1]: ```shell cd ~/git/koha-testing-docker ktd --proxy --search-engine es9 up -d ``` It is using Elasticsearch 9 (tried with 8 too). With the fixed patches I submitted. things work as expected both in staff and OPAC on both ES versions. I tested searching 'agrovoc' on the 'More options' (in both interfaces) with and without 'Apply field weights...' and 'Search entire MARC...' (the latter only applies to the staff interface). Chosen options are passed around on the facets (this was working) and now also on the 'Edit this search' and 'Return to the last advanced search' respectively. Somehow a part of the dev got lost I think. I introduced 'weight_search_submitted' to keep track of the user unchecking 'weight_search' but the template was not taking it into account. Sorry for that. Probably a bad commit. [1] You don't need to use `--proxy` for it to work.
Thanks Tomás for looking at this again! I'll have to leave re-tetsing until I've got a working environment with Elasticsearch ES8 or ES9 - hopefully someone who as working environment can test! Notes: - I've tried to retest, but I can't get KTD to work with either ES8 or ES9, only ES7. - I get similar errors to comment #10 step 5. - I think it is obviously something to do with my local environment (will attempt to setup KTD on my Windows desktop). - I also tried on the sandboxes (Open Fifth and BibLibre - errors when doing a full reindex after switching to Elasticserch; ByWater Solutions - "You are not from an authorized IP.").
Created attachment 190745 [details] [review] Bug 40966: Regression tests Tests verify that whole_record and weighted_fields parameters are correctly included in the query_cgi string returned by build_query_compat(), ensuring these parameters are preserved in facet links and pagination. To test: 1. Apply this patch 2. Run: $ ktd --shell k$ prove t/db_dependent/Koha/SearchEngine/Elasticsearch/QueryBuilder.t => FAIL: The values are not being passed around Signed-off-by: David Nind <david@davidnind.com>
Created attachment 190746 [details] [review] Bug 40966: Make sure 'whole_record' and 'weighted_fields' are passed around The ES query builder correctly handles this parameters (passed from search.pl and opac-search.pl) when it builds the ES query. But the returned 'query_cgi' that is used later for keeping the search results query context for building things like facets, just doesn't pass this values around, leading to a loss of them on the generated links. This patch makes sure they are passed around. To test: 1. Apply the regression tests 2. Run: $ ktd --shell k$ prove t/db_dependent/Koha/SearchEngine/Elasticsearch/QueryBuilder.t => FAIL: Tests fail! These params are not passed around! 3. Set ElasticsearchMARCFormat="Searchable array" 4. Reindex all the things: k$ koha-elasticsearch --rebuild -d -v kohadev 5. Go to advanced search, using the 'More options' option. 6. Check 'Search entire MARC record' and 'Apply weights to search' 7. Perform a search on 'FULL TEXT' => SUCCESS: The URL contains `whole_record=on` and `weighted_fields=on` 8. Hover on any facet and look at the link => FAIL: It doesn't contain them! 9. Apply this patch 10. Restart all k$ restart_all 11. Refresh the search results 12. Repeat 6 => SUCCESS: Links have the needed parameters! 13. Repeat 2 => SUCCESS: Tests pass! the parameters are passed around! 14. Sign off :-D Signed-off-by: David Nind <david@davidnind.com>
Got ES8 and ES9 working with KTD on a Windows desktop. Things worked as per the test plan. I used 'world' for the 'Keyword as phrase' search term.