In advanced search with Elasticsearch the limit on years range is actually using copydate : Koha/SearchEngine/Elasticsearch/QueryBuilder.pm in _fix_limit_special_cases() : if ( $l =~ /^yr,st-numeric,ge=/ ) { my ( $start, $end ) = ( $l =~ /^yr,st-numeric,ge=(.*) and yr,st-numeric,le=(.*)$/ ); next unless defined($start) && defined($end); push @new_lim, "copydate:[$start TO $end]"; } With Zebra it uses date-of-publication and also in Koha/SearchEngine/Elasticsearch/QueryBuilder.pm : our %index_field_convert = ( (...) 'yr' => 'date-of-publication',
Not sure if it helps, but for MARC21 the date range uses 008 and the publication year option int he pull down uses 260$c (maybe also 264$c, haven't checked).
This is actually in etc/zebradb/ccl.properties : #Date 30 The point of time at which 005, 008/00-05, # a transaction or event 008/07-10, 260$c, # takes place. 008/11-14, 033,etc. # interpreting this as the copyright date in 260$c (MARC21) and 210$d (UNIMARC) copydate 1=30 r=r #Date-publication 31 The date (usually year) in 008/07-10, 260$c # which a document is published. 046, 533$d Date-of-publication 1=31 r=r yr Date-of-publication pubdate Date-of-publication
Hm, so copydate = 260$c 443 <!--record.abs line 121: melm 260$c copydate,copydate:s--> 444 <index_subfields tag="260" subfields="c"> 445 <target_index>copydate:w</target_index> 446 <target_index>copydate:s</target_index> 447 </index_subfields> And pubdate 008: 49 <index_control_field tag="008" offset="7" length="4"> 50 <target_index>pubdate:w</target_index> 51 </index_control_field> and pubdate = yr = Date-of-publication ... and we are missing proper indexing of 264 (see 15704) It seems I missed a change to advanced search, could have sworn we had publication year in the drop downs too. :(
+1
Hi, Does anyone work on this problem? If not, is there a way around the problem with configuring indexes? Regards, Marjorie
Created attachment 115666 [details] [review] Bug 22801: Advance search yr must use date-of-publication and not copydate In advanced search with Elasticsearch the limit on years range is actually using copydate : Koha/SearchEngine/Elasticsearch/QueryBuilder.pm in _fix_limit_special_cases() : if ( $l =~ /^yr,st-numeric,ge=/ ) { my ( $start, $end ) = ( $l =~ /^yr,st-numeric,ge=(.*) and yr,st-numeric,le=(.*)$/ ); next unless defined($start) && defined($end); push @new_lim, "copydate:[$start TO $end]"; } With Zebra it uses date-of-publication and also in Koha/SearchEngine/Elasticsearch/QueryBuilder.pm : our %index_field_convert = ( (...) 'yr' => 'date-of-publication', This patch uses %index_field_convert to perform 'yr' limit. Test plan: 1) Apply patch 2) Use Elasticsearch searchengine 3) Go to advanced search with 'More options' 4) Perform a search with a year limit (value or range) 5) Check results are correct
Created attachment 122064 [details] [review] Bug 22801: Advance search yr must use date-of-publication and not copydate In advanced search with Elasticsearch the limit on years range is actually using copydate : Koha/SearchEngine/Elasticsearch/QueryBuilder.pm in _fix_limit_special_cases() : if ( $l =~ /^yr,st-numeric,ge=/ ) { my ( $start, $end ) = ( $l =~ /^yr,st-numeric,ge=(.*) and yr,st-numeric,le=(.*)$/ ); next unless defined($start) && defined($end); push @new_lim, "copydate:[$start TO $end]"; } With Zebra it uses date-of-publication and also in Koha/SearchEngine/Elasticsearch/QueryBuilder.pm : our %index_field_convert = ( (...) 'yr' => 'date-of-publication', This patch uses %index_field_convert to perform 'yr' limit. Test plan: 1) Apply patch 2) Use Elasticsearch searchengine 3) Go to advanced search with 'More options' 4) Perform a search with a year limit (value or range) 5) Check results are correct Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Created attachment 122065 [details] [review] Bug 22801: (follow-up) Use date-of-publication directly Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Hi Frido, I added a followup to remove: my $yrfield = ( exists $index_field_convert{'yr'} ) ? $index_field_convert{'yr'} : 'yr'; The index_field_convert will always be built the same, so will always resolve to date-of-publication - so we don't need the condition, right? I also update tests to pass
Created attachment 122066 [details] [review] Bug 22801: Advance search yr must use date-of-publication and not copydate In advanced search with Elasticsearch the limit on years range is actually using copydate : Koha/SearchEngine/Elasticsearch/QueryBuilder.pm in _fix_limit_special_cases() : if ( $l =~ /^yr,st-numeric,ge=/ ) { my ( $start, $end ) = ( $l =~ /^yr,st-numeric,ge=(.*) and yr,st-numeric,le=(.*)$/ ); next unless defined($start) && defined($end); push @new_lim, "copydate:[$start TO $end]"; } With Zebra it uses date-of-publication and also in Koha/SearchEngine/Elasticsearch/QueryBuilder.pm : our %index_field_convert = ( (...) 'yr' => 'date-of-publication', This patch uses %index_field_convert to perform 'yr' limit. Test plan: 1) Apply patch 2) Use Elasticsearch searchengine 3) Go to advanced search with 'More options' 4) Perform a search with a year limit (value or range) 5) Check results are correct Signed-off-by: David Nind <david@davidnind.com>
(In reply to David Nind from comment #10) > Signed-off-by: David Nind <david@davidnind.com> Looks like we overlapped, can you test with second patch as well David?
> Looks like we overlapped, can you test with second patch as well David? Will do!
Created attachment 122069 [details] [review] Bug 22801: Advance search yr must use date-of-publication and not copydate In advanced search with Elasticsearch the limit on years range is actually using copydate : Koha/SearchEngine/Elasticsearch/QueryBuilder.pm in _fix_limit_special_cases() : if ( $l =~ /^yr,st-numeric,ge=/ ) { my ( $start, $end ) = ( $l =~ /^yr,st-numeric,ge=(.*) and yr,st-numeric,le=(.*)$/ ); next unless defined($start) && defined($end); push @new_lim, "copydate:[$start TO $end]"; } With Zebra it uses date-of-publication and also in Koha/SearchEngine/Elasticsearch/QueryBuilder.pm : our %index_field_convert = ( (...) 'yr' => 'date-of-publication', This patch uses %index_field_convert to perform 'yr' limit. Test plan: 1) Apply patch 2) Use Elasticsearch searchengine 3) Go to advanced search with 'More options' 4) Perform a search with a year limit (value or range) 5) Check results are correct Signed-off-by: David Nind <david@davidnind.com>
Created attachment 122070 [details] [review] Bug 22801: (follow-up) Use date-of-publication directly Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: David Nind <david@davidnind.com>
Testing notes (koha-testing-docker): - tested first using Zebra: works as expected - changed search engine to Elastic and reindexed - change a record so that 008 and 260$c have different values (such as 2009 and 2011) - use the advanced search form > more options > limits > year to search on the 008 value (such as 2009) => record will NOT appear in the search results - search using the 260$c value (such as 2011) => record WILL appear in the search results - apply the patch (and reindex - not sure whether this is necessary, but did anyway) - correct behavour should now occur: . results for limit search should show matching record with 008 value, not 260$c value
Created attachment 123138 [details] [review] Bug 22801: Advance search yr must use date-of-publication and not copydate In advanced search with Elasticsearch the limit on years range is actually using copydate : Koha/SearchEngine/Elasticsearch/QueryBuilder.pm in _fix_limit_special_cases() : if ( $l =~ /^yr,st-numeric,ge=/ ) { my ( $start, $end ) = ( $l =~ /^yr,st-numeric,ge=(.*) and yr,st-numeric,le=(.*)$/ ); next unless defined($start) && defined($end); push @new_lim, "copydate:[$start TO $end]"; } With Zebra it uses date-of-publication and also in Koha/SearchEngine/Elasticsearch/QueryBuilder.pm : our %index_field_convert = ( (...) 'yr' => 'date-of-publication', This patch uses %index_field_convert to perform 'yr' limit. Test plan: 1) Apply patch 2) Use Elasticsearch searchengine 3) Go to advanced search with 'More options' 4) Perform a search with a year limit (value or range) 5) Check results are correct Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Created attachment 123139 [details] [review] Bug 22801: (follow-up) Use date-of-publication directly Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Pushed to master for 21.11, thanks to everybody involved!
Pushed to 21.05.x fro 21.05.03
Pushed to 20.11.x for 20.11.09 Viva Elasticsearch ^^
Missing dependencies for 20.05.x, it shouldn't be affected, no backport.
errata: no issue related to dependencies, anyway: Not backported to oldoldstable (20.05.x). Feel free to ask if it's needed.