Bug 24438 - index can fail on timeout
Summary: index can fail on timeout
Status: NEW
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching - Elasticsearch (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal
Assignee: Bugs List
QA Contact:
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-01-16 18:14 UTC by Nick Clemens (kidclamp)
Modified: 2023-12-09 17:19 UTC (History)
3 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Clemens (kidclamp) 2020-01-16 18:14:46 UTC
We have seen occasional timeouts when indexing elasticsearch, these cause the indexing to stop and fail.

We should handle the response as the code indicates:
102 sub update_index {
103     my ($self, $biblionums, $records) = @_;
104 
105     my $conf = $self->get_elasticsearch_params();
106     my $elasticsearch = $self->get_elasticsearch();
107     my $documents = $self->marc_records_to_documents($records);
108     my @body;
109 
110     for (my $i=0; $i < scalar @$biblionums; $i++) {
111         my $id = $biblionums->[$i];
112         my $document = $documents->[$i];
113         push @body, {
114             index => {
115                 _id => $id
116             }
117         };
118         push @body, $document;
119     }
120     if (@body) {
121         my $response = $elasticsearch->bulk(
122             index => $conf->{index_name},
123             type => 'data', # is just hard coded in Indexer.pm?
124             body => \@body
125         );
126     }
127     # TODO: handle response
128     return 1;
129 }
Comment 1 Björn Nylén 2020-11-11 10:28:19 UTC
WE've been seeing the same. However we've recently tried to add a request_timeout parameter the the ES indexer in rebuild_elasticsearch.pl to work around the problem.
Comment 2 Katrin Fischer 2023-12-09 17:19:18 UTC
3 years later - does this problem still exist in our current implementation?