Bug 9354 - Add ability to validate MARC records before import
Summary: Add ability to validate MARC records before import
Status: NEW
Alias: None
Product: Koha
Classification: Unclassified
Component: Cataloging (show other bugs)
Version: Main
Hardware: All All
: P5 - low new feature
Assignee: Galen Charlton
QA Contact:
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-01-07 01:49 UTC by Doug Kingston
Modified: 2022-08-22 02:36 UTC (History)
3 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Doug Kingston 2013-01-07 01:49:54 UTC
I think this is a MAJOR bug because it leads to silent data corruption.  You can reclassify as an enhancement if you don't agree with this assessment.

The Koha MARC import functions do little if any MARC record validation and some errors in MARC formatting can lead to bad data imports with no warning.  Case in point for us was generating records with pymarc (with Python) and not setting the 'force_utf8' flag.  We generated records that appeared valid, but tools like MarcEdit clearly complain about the format (length errors, and leader errors).
leader[9] should have been an 'a', indicating that the data should be interpreted as UTF8.  Since it was missing, the data fields appeared truncated because the MARC offset calculations got screwed up.

Koha said nothing.  We should incorporate the same validation checks that MarcEdit makes to flag bad import files before the data is accepted.
Comment 1 Jared Camins-Esakov 2013-01-07 13:08:58 UTC
This is an intriguing feature idea, and I hope someone decides to implement it. That said, it is a not a bug because it does not result in the loss of valid data. It sounds like pymarc may have a bug, though. The LDR field is a fixed length, regardless whether LDR/09 is set.
Comment 2 David Cook 2022-08-22 02:36:31 UTC
This would be interesting to add now that we're moving the MARC record import to background tasks...