Summary: | Problem when importing only items in MARC records | ||
---|---|---|---|
Product: | Koha | Reporter: | Fridolin Somers <fridolin.somers> |
Component: | Tools | Assignee: | Fridolin Somers <fridolin.somers> |
Status: | CLOSED FIXED | QA Contact: | Testopia <testopia> |
Severity: | normal | ||
Priority: | P5 - low | CC: | alexbuckley, dubyk, giuseppe.ciaccio, martin.renvoize, mtompset, sandboxes, tomascohen, viliam.polonec |
Version: | Main | ||
Hardware: | All | ||
OS: | All | ||
See Also: | https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17240 | ||
GIT URL: | Change sponsored?: | --- | |
Patch complexity: | Trivial patch | Documentation contact: | |
Documentation submission: | Text to go in the release notes: | ||
Version(s) released in: |
19.11.00
|
Circulation function: | |
Attachments: |
Bug 18757 - Problem when importing only items in MARC records
Bug 18757 - Problem when importing only items in MARC records Bug 18757 - Problem when importing only items in MARC records |
Description
Fridolin Somers
2017-06-08 14:21:44 UTC
Created attachment 64104 [details] [review] Bug 18757 - Problem when importing only items in MARC records When importing records with Stage MARC records for import, one can use matching rules to only import items into existing records. Those imported items are stored as XML to be staged. The bug is that when MARC Flavour is UNIMARC the XML serialization fails because its is looking in field 100$a which does not exist. You see in logs the error : Unsupported UNIMARC character encoding [] for XML output for UNIMARC; 100$a This patch adds the format "USMARC" to XML serialization, like in C4::Items::_get_unlinked_subfields_xml Test plan : - On a UNIMARC database - Define a maching rule on title 200$a - Select a record with items - Export it using : Save as > MARC (Unicode/UTF-8) - Delete all items - Go to Tools > Stage MARC records for import - Upload exported file - Select title matching rule - Select "Ingore incoming record" in "Action if matching record found :" - Select Yes and "Always add items" in "Check for embedded item record data?" - Click Stage for import => Without patch you get the error => With patch the import is staged - Import into the catalog and check item is well recreated Hi Fridolin When applying your patch on a clean branch up to date with master, I find that I am still getting the error message 'Unsupported UNIMARC character encoding [] for XML output for UNIMARC; 100$a' as I do without the patch applied. I am getting this error when trying to both create and import records. I have tried setting the matching record rule to both 100$a and 200$a but continue to get the same error. Looks like this only occurs with Plack. One must test with plack on tools/manage-marc-import.pl. Since there are problems with background jobs with plack, test with Bug 17240. But this patch is I think logic since it does XML serialization like in C4::Items::_get_unlinked_subfields_xml (In reply to Alex Buckley from comment #2) > Hi Fridolin > > When applying your patch on a clean branch up to date with master, I find > that I am still getting the error message 'Unsupported UNIMARC character > encoding [] for XML output for UNIMARC; 100$a' as I do without the patch > applied. > > I am getting this error when trying to both create and import records. > > I have tried setting the matching record rule to both 100$a and 200$a but > continue to get the same error. Make sure you restart Plack after applying the patch. I set back to NSO https://archive.ifla.org/VI/8/projects/UNIMARC-HoldingsFormat.pdf -- but isn't there supposed to be a 100?! 12-15 Character Set (Mandatory) These four character positions indicate the principal graphic character sets used in the record. Positions 12-13 designate the G0 set and positions 14-15 designate the G1 set. If a G1 set is not needed, positions 14-15 contain blanks. 01 = ISO 646, IRV version (basic Latin set) 02 = ISO Registration # 37 (basic Cyrillic set) 03 = ISO 5426 (extended Latin set) 04 = ISO DIS 5427 (extended Cyrillic set) 05 = ISO 5428 (Greek set) 06 = ISO 6438 (African coded character set) 07 = ISO 10586 (Georgian set) 08 = ISO 8957 (Hebrew set) Table 1 09 = ISO 8957 (Hebrew set) Table 2 10 = [Reserved] 11 = ISO 5426-2 (Latin characters used in minor European languages and obsolete typography) 50 = ISO 10646 Level 3 (Unicode) Oh yuck... reading the MARC::File::XML library... it basically fakes it. # Does the record think it is already Unicode? if ($original_encoding ne 'a' && lc($format) !~ /^unimarc/o) { # If not, we'll make it so $_transcode++; substr($ldr,9,1,'a'); $record->leader( $ldr ); } which is why the as_xml in elsewhere does the "USMARC" thing. Okay. I'll test this, since I'm assuming Fridolin has already tested in a UNIMARC context before submitting this patch. BTW, it should be noted that this same problem shows itself as an "Internal Server Error" if one tries to manually catalogue a record in UNIMARC, and uses something clearly bad, like "KABOOM!" in the 100$a field. :) substr outside of string at /usr/share/perl5/MARC/File/XML.pm line 561 (#2) (W substr)(F) You tried to reference a substr() that pointed outside of a string. That is, the absolute value of the offset was larger than the length of the string. See "substr" in perlfunc. This warning is fatal if substr is used in an lvalue context (as the left hand side of an assignment or as a subroutine argument for example). Use of uninitialized value $enc in string eq at /usr/share/perl5/MARC/File/XML.pm line 563 (#1) Use of uninitialized value $enc in string eq at /usr/share/perl5/MARC/File/XML.pm line 565 (#1) Use of uninitialized value $enc in concatenation (.) or string at /usr/share/perl5/MARC/File/XML.pm line 568 (#1) Unsupported UNIMARC character encoding [] for XML output for UNIMARC; 100$a -> kaboom at /usr/share/perl5/MARC/File/XML.pm line 568. No need to make any 200 rules. (In reply to M. Tompsett from comment #7) > which is why the as_xml in elsewhere does the "USMARC" thing. Okay. I'll > test this, since I'm assuming Fridolin has already tested in a UNIMARC > context before submitting this patch. Indeed. Several of our Koha UNIMARC have already this patch. UNIMARC 100$a is a spaghetti dish ;-) Patch tested with a sandbox, by Amandine Zocca <azocca@ville-montauban.fr> Created attachment 93363 [details] [review] Bug 18757 - Problem when importing only items in MARC records When importing records with Stage MARC records for import, one can use matching rules to only import items into existing records. Those imported items are stored as XML to be staged. The bug is that when MARC Flavour is UNIMARC the XML serialization fails because its is looking in field 100$a which does not exist. You see in logs the error : Unsupported UNIMARC character encoding [] for XML output for UNIMARC; 100$a This patch adds the format "USMARC" to XML serialization, like in C4::Items::_get_unlinked_subfields_xml Test plan : - On a UNIMARC database - Define a maching rule on title 200$a - Select a record with items - Export it using : Save as > MARC (Unicode/UTF-8) - Delete all items - Go to Tools > Stage MARC records for import - Upload exported file - Select title matching rule - Select "Ingore incoming record" in "Action if matching record found :" - Select Yes and "Always add items" in "Check for embedded item record data?" - Click Stage for import => Without patch you get the error => With patch the import is staged - Import into the catalog and check item is well recreated Signed-off-by: Amandine Zocca <azocca@ville-montauban.fr> Created attachment 94826 [details] [review] Bug 18757 - Problem when importing only items in MARC records When importing records with Stage MARC records for import, one can use matching rules to only import items into existing records. Those imported items are stored as XML to be staged. The bug is that when MARC Flavour is UNIMARC the XML serialization fails because its is looking in field 100$a which does not exist. You see in logs the error : Unsupported UNIMARC character encoding [] for XML output for UNIMARC; 100$a This patch adds the format "USMARC" to XML serialization, like in C4::Items::_get_unlinked_subfields_xml Test plan : - On a UNIMARC database - Define a maching rule on title 200$a - Select a record with items - Export it using : Save as > MARC (Unicode/UTF-8) - Delete all items - Go to Tools > Stage MARC records for import - Upload exported file - Select title matching rule - Select "Ingore incoming record" in "Action if matching record found :" - Select Yes and "Always add items" in "Check for embedded item record data?" - Click Stage for import => Without patch you get the error => With patch the import is staged - Import into the catalog and check item is well recreated Signed-off-by: Amandine Zocca <azocca@ville-montauban.fr> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Works as expected, passes qa scripts and code looks sound.. Passing QA Nice work! Pushed to master for 19.11.00 *** Bug 23684 has been marked as a duplicate of this bug. *** |