Bug 26522 - Elasticsearch - don't attempt to index record if it cannot be retrieved
Summary: Elasticsearch - don't attempt to index record if it cannot be retrieved
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching - Elasticsearch (show other bugs)
Version: Main
Hardware: All All
: P5 - low major (vote)
Assignee: Nick Clemens (kidclamp)
QA Contact:
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-09-23 17:57 UTC by Nick Clemens (kidclamp)
Modified: 2021-12-13 21:09 UTC (History)
1 user (show)

See Also:
Change sponsored?: ---
Patch complexity: Trivial patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
Bug 26522: Don't index record if no record is retrieved (1.58 KB, patch)
2020-09-23 18:52 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Clemens (kidclamp) 2020-09-23 17:57:45 UTC
In ModZebra we call GetMarcBiblio - this can fail on malformed records.

If it does, we should not attempt to index, otherwsie we get an error:
[Wed Sep 23 14:49:59.978945 2020] [cgi:error] [pid 131451] [client 172.18.0.1:46768] AH01215: [Wed Sep 23 14:49:59 2020] manage-marc-import.pl: {UNKNOWN}: Can't call method "leader" on an undefined value at /kohadevbox/koha/Koha/SearchEngine/Elasticsearch.pm line 566. at /kohadevbox/koha/tools/manage-marc-import.pl line 252: /kohadevbox/koha/tools/manage-marc-import.pl, referer: http://localhost:8081/cgi-bin/koha/tools/manage-marc-import.pl?import_batch_id=2
Comment 1 Nick Clemens (kidclamp) 2020-09-23 18:52:22 UTC
Created attachment 110614 [details] [review]
Bug 26522: Don't index record if no record is retrieved

Currently if we cannot load the marc record, we die. This patch simply skips
the malformed record during indexing

Eventually we should warn or pass back an exception, but we can use
compare_es_to_db.pl to identify problematic records and should fix this
issue to prevent import from dying

To test:
1 - Export or create a bib with malformed data
    In sample data search for 'udgave' the record has
    942             _6_
2 - Set System preference SearchEngine to Elasticsearch
3 - Import the record
4 - Import fails
5 - Check intranet-error.log, see warning about calling leader on undefined value
6 - Apply patch
7 - Import again
8 - Success, import succeeds
9 - Search for 'udgave' or term that returns your record
10 - The record should not be findable
Comment 2 Victor Grousset/tuxayo 2020-11-12 14:56:26 UTC
Applying: Bug 26522: Don't index record if no record is retrieved
error: sha1 information is lacking or useless (C4/Biblio.pm).
error: could not build fake ancestor
Patch failed at 0001 Bug 26522: Don't index record if no record is retrieved
Comment 3 Victor Grousset/tuxayo 2020-11-12 15:05:09 UTC
> 4 - Import fails

I was able to stage and import according to the UI:
> Number of records added 1


> 5 - Check intranet-error.log, see warning about calling leader on undefined value

Nothing such in /var/log/koha/kohadev/intranet-error.log
As well as /var/log/koha/kohadev/plack-error.log
Comment 4 Victor Grousset/tuxayo 2020-11-12 20:52:39 UTC
As well as /var/log/koha/kohadev/plack-intranet-error.log

(got confused with the log files)
Comment 5 Nick Clemens (kidclamp) 2021-01-19 15:01:22 UTC
Bug 25265 fixed this I believe