Bug 6433 - rebuild_zebra.pl does not handle MARC::Record exceptions gracefully
Summary: rebuild_zebra.pl does not handle MARC::Record exceptions gracefully
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Searching (show other bugs)
Version: 3.6
Hardware: All All
: PATCH-Sent (DO NOT USE) enhancement (vote)
Assignee: Srdjan Jankovic
QA Contact: Bugs List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-05-31 03:16 UTC by Srdjan Jankovic
Modified: 2013-12-05 19:53 UTC (History)
2 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
patch (3.43 KB, patch)
2011-05-31 03:37 UTC, Srdjan Jankovic
Details | Diff | Splinter Review
Signed-off patch (3.50 KB, patch)
2011-05-31 12:08 UTC, Magnus Enger
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Srdjan Jankovic 2011-05-31 03:16:34 UTC
rebuild_zebra.pl dies on some MARC::Record errors. A combination of workarounds for MARC::Record deficiencies and legitimate exception handling.
Comment 1 Srdjan Jankovic 2011-05-31 03:37:48 UTC Comment hidden (obsolete)
Comment 2 Magnus Enger 2011-05-31 12:08:44 UTC
Created attachment 4318 [details] [review]
Signed-off patch

This patch solved a real problem for me. I was rebuilding Zebra and getting this: 

11401...........................................................................
11501...........................................................................
11601.................Wide character in subroutine entry at /usr/share/perl5/MARC/Charset/Table.pm line 96.

After applying the patch I got this, which is a lot easier to debug: 

11601...Error exporting record 13611 (biblio) XML at misc/migration_tools/rebuild_zebra.pl line 359.
.....................................................Error exporting record 13665 (biblio) XML at misc/migration_tools/rebuild_zebra.pl line 359.
..........................................Error exporting record 13708 (biblio) XML at misc/migration_tools/rebuild_zebra.pl line 359.

I do have one minor issue with how rebuild_zebra,pl works now: After applying the patch the script is more likely to survive bad records, but when there are a lot of records, Zebra's listing of the tmp files for the first 1000 records indexed (do we need that anyway?) will fill the scrollback and make it impossible to go back and see if there were any errors during indexing. This can be worked around by redirecting output from the script to a file which can be inspected at leisure after the script has run it's course: 

rebuild_zebra.pl -b -v &> indexlog.txt

But perhaps it would be possible to give a summary of any errors after the last of the output that the script produces today? 

I'm signing off anyway, 'cause I think this is an improvement over the current script.
Comment 3 Chris Cormack 2011-06-09 23:31:09 UTC
Tested and works well, pushed to master
Comment 4 Jared Camins-Esakov 2012-05-23 12:20:34 UTC
So far as it goes, this seems to work. Closing.