Bug 9354

Summary: Add ability to validate MARC records before import
Product: Koha Reporter: Doug Kingston <dpk>
Component: CatalogingAssignee: Galen Charlton <gmcharlt>
Status: NEW --- QA Contact:
Severity: new feature    
Priority: P5 - low CC: dcook, josef.moravec, katrin.fischer, m.de.rooy
Version: Main   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27421
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:

Description Doug Kingston 2013-01-07 01:49:54 UTC
I think this is a MAJOR bug because it leads to silent data corruption.  You can reclassify as an enhancement if you don't agree with this assessment.

The Koha MARC import functions do little if any MARC record validation and some errors in MARC formatting can lead to bad data imports with no warning.  Case in point for us was generating records with pymarc (with Python) and not setting the 'force_utf8' flag.  We generated records that appeared valid, but tools like MarcEdit clearly complain about the format (length errors, and leader errors).
leader[9] should have been an 'a', indicating that the data should be interpreted as UTF8.  Since it was missing, the data fields appeared truncated because the MARC offset calculations got screwed up.

Koha said nothing.  We should incorporate the same validation checks that MarcEdit makes to flag bad import files before the data is accepted.
Comment 1 Jared Camins-Esakov 2013-01-07 13:08:58 UTC
This is an intriguing feature idea, and I hope someone decides to implement it. That said, it is a not a bug because it does not result in the loss of valid data. It sounds like pymarc may have a bug, though. The LDR field is a fixed length, regardless whether LDR/09 is set.
Comment 2 David Cook 2022-08-22 02:36:31 UTC
This would be interesting to add now that we're moving the MARC record import to background tasks...