Bug 32920 - ES indexer should gracefully handle a bad bib record when reindexing after a batch modification
Summary: ES indexer should gracefully handle a bad bib record when reindexing after a ...
Status: In Discussion
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching - Elasticsearch (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal
Assignee: Bugs List
QA Contact:
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-02-08 21:42 UTC by Emily Lamancusa (emlam)
Modified: 2024-11-06 22:52 UTC (History)
4 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Emily Lamancusa (emlam) 2023-02-08 21:42:58 UTC
When Elasticsearch reindexes after a batch modification, if one of the modified biblio records contains invalid data, the "Update Elasticsearch Index" job will fail without updating the search indexing for any of the modified records. When reindexing multiple records at once, an error with one record should not prevent the others from being reindexed correctly.

To reproduce:
1. Set SearchEngine system preference to 'Elasticsearch'
2. Find or produce a biblio record that contains invalid data
   (in KTD's sample data, biblionumber 369 works for this)
3. Confirm that the biblio record detail page in the staff interface displays
   an invalid data error
4. Generate a barcode file that includes items attached to this biblio as well as other items
5. Perform a batch item modification
6. Open the job list. Note "Finished" status for batch item mod job, and "Failed" status for Update Elasticsearch index job
7. Find the modified records via catalog search. Note that none of the modified items display the updated information in the catalog search.

I reproduced this error on KTD, by doing a batch edit on all items with publication date 198_

In case it's useful, error message from the failed job:

^ at /kohadevbox/koha/Koha/Biblio/Metadata.pm line 114.
DEBUG - Update of elastic index failed with: Invalid data, cannot decode metadata object (biblio_metadata.id=368, biblionumber=369, format=marcxml, schema=MARC21, decoding_error=':8: parser error : PCDATA invalid Char value 31
  <controlfield tag="001">00aD000015937</controlfield>
                            ^
:9: parser error : PCDATA invalid Char value 31
  <controlfield tag="004">00satmrnu0</controlfield>
                            ^
:9: parser error : PCDATA invalid Char value 31
  <controlfield tag="004">00satmrnu0</controlfield>
                               ^
:9: parser error : PCDATA invalid Char value 31
  <controlfield tag="004">00satmrnu0</controlfield>
                                  ^
:9: parser error : PCDATA invalid Char value 31
  <controlfield tag="004">00satmrnu0</controlfield>
                                     ^
:10: parser error : PCDATA invalid Char value 31
  <controlfield tag="008">00ar19881981bdkldan</controlfield>
                            ^
:10: parser error : PCDATA invalid Char value 31
  <controlfield tag="008">00ar19881981bdkldan</controlfield>
                                       ^
:10: parser error : PCDATA invalid Char value 31
  <controlfield tag="008">00ar19881981bdkldan</controlfield>
                                           ^')
Comment 1 Jonathan Druart 2023-02-09 08:37:17 UTC
This is not an easy one.

We could eventually have the 2 jobs linked together (the batch job would be aware of an index job that it triggered), but that is not trivial to implement.