Bug 30522 - SearchEngine::simple_search_compat should not limit to 100 records for ElasticSearch (causing incomplete authority merges)
Summary: SearchEngine::simple_search_compat should not limit to 100 records for Elasti...
Status: RESOLVED DUPLICATE of bug 30883
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching - Elasticsearch (show other bugs)
Version: master
Hardware: All All
: P5 - low major (vote)
Assignee: Bugs List
QA Contact:
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-04-13 11:30 UTC by Bernard
Modified: 2022-06-06 21:54 UTC (History)
4 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Bernard 2022-04-13 11:30:24 UTC
With systempreference AuthorityMergeLimit set to say 50, if you amend an authority record which has more than 100 linked bibs and run merge_authorities -b -v - then only the first 100 bibs are amended to match the authority. The remaining ones are unmodified.

I think it relates to line 81 of /Koha/Authorities.pm and what is returned to the variable  $result.
Comment 1 Marcel de Rooy 2022-04-13 12:10:39 UTC
Interesting. Could provide details about the version you experience this?

I tested this:

use Modern::Perl;
use Data::Dumper qw/Dumper/;
use Koha::Authorities;
my @biblionumbers = Koha::Authorities->linked_biblionumbers({ authid => 1 });
print Dumper( scalar @biblionumbers );

I have 149 biblionumbers attached to authid 1.
And got this result:

$VAR1 = 149;

So that should be fine (as a first check).
Comment 2 Marcel de Rooy 2022-04-13 12:15:10 UTC
But I am seeing something funny.
From my 149 records, 148 records have been updated when I made a change in the authority. But one record (somewhere in the middle) has not been updated!

Note that 148 > 100 records and this tells me that your problem does not occur in current master. And I do think that 21.11 and master are not different as to merging authorities.

Could you clarify/show more details about your conclusion that only 100 records were updated in your catalog? How do you reach that conclusion?
Comment 3 Marcel de Rooy 2022-04-13 12:31:57 UTC
(In reply to Marcel de Rooy from comment #2)
> From my 149 records, 148 records have been updated when I made a change in
> the authority. But one record (somewhere in the middle) has not been updated!

This must have been something else. From a previous test. Created with build_sample_biblio.

I removed all those records. Created 200 records. Linked to authid 1.
Changed authid 1. All 200 records are merged by the cron script.
All working fine.
Comment 4 Bernard 2022-04-13 13:09:21 UTC
It's version 21.11.01.000

I've just run your little script:

use Modern::Perl;
use Data::Dumper qw/Dumper/;
use Koha::Authorities;
my @biblionumbers = Koha::Authorities->linked_biblionumbers({ authid => 7307 });
print Dumper( scalar @biblionumbers );

>>>

$ perl /home2/kfc/kohaclone/Koha/bertest.pl
$VAR1 = 100;

There are 104 bib records linked to this authority
Comment 5 Marcel de Rooy 2022-04-13 14:09:42 UTC
(In reply to Bernard from comment #4)
> It's version 21.11.01.000
> 
> I've just run your little script:
> 
> use Modern::Perl;
> use Data::Dumper qw/Dumper/;
> use Koha::Authorities;
> my @biblionumbers = Koha::Authorities->linked_biblionumbers({ authid => 7307
> });
> print Dumper( scalar @biblionumbers );
> 
> >>>
> 
> $ perl /home2/kfc/kohaclone/Koha/bertest.pl
> $VAR1 = 100;
> 
> There are 104 bib records linked to this authority

Found it!
You are using ElasticSearch, I used Zebra.
The cause is in the Koha/SearchEngine/ElasticSearch/Search.pm module:

sub simple_search_compat {
    my ($self, $query, $offset, $max_results) = @_;
    return ('No query entered', undef, undef) unless $query;
    my %options;
    $offset = 0 if not defined $offset or $offset < 0;
    $options{offset} = $offset;
    $max_results //= 100;

=> This sub expects a max_results parameter for Elastic while the Zebra one does not require it. No max_results means there all results.
So we need to adjust that one somehow. CCing Nick and Tomas.

Good catch !
Comment 6 Katrin Fischer 2022-06-06 21:54:14 UTC
Bad news: this is a duplicate now. Good news: the duplicate bug has a patch and is waiting for QA.

*** This bug has been marked as a duplicate of bug 30883 ***