Bug 6433

Summary: rebuild_zebra.pl does not handle MARC::Record exceptions gracefully
Product: Koha Reporter: Srdjan Jankovic <srdjan>
Component: SearchingAssignee: Srdjan Jankovic <srdjan>
Status: CLOSED FIXED QA Contact: Bugs List <koha-bugs>
Severity: enhancement    
Priority: PATCH-Sent (DO NOT USE) CC: chris, magnus
Version: 3.6   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:
Attachments: patch
Signed-off patch

Description Srdjan Jankovic 2011-05-31 03:16:34 UTC
rebuild_zebra.pl dies on some MARC::Record errors. A combination of workarounds for MARC::Record deficiencies and legitimate exception handling.
Comment 1 Srdjan Jankovic 2011-05-31 03:37:48 UTC Comment hidden (obsolete)
Comment 2 Magnus Enger 2011-05-31 12:08:44 UTC
Created attachment 4318 [details] [review]
Signed-off patch

This patch solved a real problem for me. I was rebuilding Zebra and getting this: 

11401...........................................................................
11501...........................................................................
11601.................Wide character in subroutine entry at /usr/share/perl5/MARC/Charset/Table.pm line 96.

After applying the patch I got this, which is a lot easier to debug: 

11601...Error exporting record 13611 (biblio) XML at misc/migration_tools/rebuild_zebra.pl line 359.
.....................................................Error exporting record 13665 (biblio) XML at misc/migration_tools/rebuild_zebra.pl line 359.
..........................................Error exporting record 13708 (biblio) XML at misc/migration_tools/rebuild_zebra.pl line 359.

I do have one minor issue with how rebuild_zebra,pl works now: After applying the patch the script is more likely to survive bad records, but when there are a lot of records, Zebra's listing of the tmp files for the first 1000 records indexed (do we need that anyway?) will fill the scrollback and make it impossible to go back and see if there were any errors during indexing. This can be worked around by redirecting output from the script to a file which can be inspected at leisure after the script has run it's course: 

rebuild_zebra.pl -b -v &> indexlog.txt

But perhaps it would be possible to give a summary of any errors after the last of the output that the script produces today? 

I'm signing off anyway, 'cause I think this is an improvement over the current script.
Comment 3 Chris Cormack 2011-06-09 23:31:09 UTC
Tested and works well, pushed to master
Comment 4 Jared Camins-Esakov 2012-05-23 12:20:34 UTC
So far as it goes, this seems to work. Closing.