With ElasticSearch 8 I get the following error while indexing: ``` Can't locate object method "type" via package "UTF-8 "\xC3" does not map to Unicode at /usr/share/perl5/MARC/File/Encode.pm line 35. " (perhaps you forgot to load "UTF-8 "\xC3" does not map to Unicode at /usr/share/perl5/MARC/File/Encode.pm line 35. "?) at /usr/share/koha/bin/search_tools/rebuild_elasticsearch.pl line 321. Something went wrong rebuilding indexes for ... ``` I am not used to perl, at first I thought I had some weird encoding in my record, but now I think this is actually a bug in the indexing script (rebuild_elasticsearch.pl line 321), which looks like this: (...) while ( my $record = $next->() ) { my $id = $record->id // $record->authid; my $record = $record->record; $count++; if ( $verbose == 1 ) { _log( 1, "$count records processed\n" ) if ( $count % 1000 == 0); } else { _log( 2, "$id\n" ); } push @id_buffer, $id; push @commit_buffer, $record; if ( !( --$commit_count ) ) { _log( 1, "Committing $commit records...\n" ); my $response; try{ $response = $indexer->update_index( \@id_buffer, \@commit_buffer ); _handle_response($response); _log( 1, "Commit complete\n" ); } catch { _log(1,"Elasticsearch exception thrown: ".$_->type."\n"); _log(2,"Details: ".$_->details."\n"); }; $commit_count = $commit; @id_buffer = (); @commit_buffer = (); } } (...) So the catch log is blowing up the indexing process. I can get the script to process most records by changing those two logs in the catch-block, but I am unsure how to fix the logging to see what the underlying issue with my records is.
At fist glance it looks very similar to the issue solved by Bug 38913. Could you please verify and maybe confirm that the above mentioned patch fixes your issue?
Yeah this sounds like it could be a duplicate of bug 38913
bug 38913 will be in the not yet released 24.11.02
Yes, this is a duplicate. Thank you for pointing me toward the other ticket. :)
*** This bug has been marked as a duplicate of bug 38913 ***