Bug 4226 - bulkmarcimport.pl doesn't replace 001 when keepids is not specified
Summary: bulkmarcimport.pl doesn't replace 001 when keepids is not specified
Status: CLOSED INVALID
Alias: None
Product: Koha
Classification: Unclassified
Component: Command-line Utilities (show other bugs)
Version: Main
Hardware: PC All
: P5 - low normal with 5 votes (vote)
Assignee: Bugs List
QA Contact: Bugs List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-02-18 09:20 UTC by Chris Cormack
Modified: 2023-12-28 20:44 UTC (History)
5 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
An utility to complement bulkmarcimport.pl (10.86 KB, patch)
2011-12-02 10:00 UTC, Stefano Bargioni
Details | Diff | Splinter Review
Bug 4226 : New script to keep 001 numbers when importing (11.77 KB, patch)
2012-07-01 01:07 UTC, Chris Cormack
Details | Diff | Splinter Review
Bug 4226: Follow to address comment #9 (19.09 KB, patch)
2015-10-19 18:55 UTC, Mark Tompsett
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Chris Cormack 2010-05-21 01:24:38 UTC


---- Reported by brian@jhu.edu 2010-02-18 21:20:21 ----

When loading authorities, bulkmarcimport should replace the value in 001 with the Koha authid.  If you use keepids, the 001 is deleted and replaced. Without keepids, the script attempts to reuse the 001 as the Koha authid.  I'm submitting a patch which checks the 003 of MARC21 records and doesn't attempt to use the authid of the record unless it matches the MARCOrgCode, since you probably shouldn't re-use authids that aren't from your system.  I wasn't able to find a UNIMARC equivalent to check.  

It seems to me that the whole attempt to re-use the 001 as the authid should simply be deleted, since it ends up attempting to insert it into an auto_increment field in the DB.  Is there something I'm missing here?



---- Additional Comments From nicolas.morin@biblibre.com 2010-02-22 09:43:47 ----

One might want to use keepids not to keep the original 001 as the internal authid, but as an information in another fields.
In Unimarc you might want to do bulkmarcimport.pl --keepids 035 for instance, moving the original 001 to 035 and letting Koha create it's own 001 as authid.

So the -keepids option should remain. That being said I agree that trying to reuse the incoming ID as internal authid is not recommended.



---- Additional Comments From katrin.fischer@bsz-bw.de 2010-02-22 10:35:59 ----

I think your suggestion might break our current solution for importing bibiographic and authority data from our union catalog.

All cataloging in our libraries is done in the union catlaog. We import changes and new data into Koha every night.

The auth number in bibliographic data and auth number of authority must match, so they get linked to each other correctly.

When an authority is assigned an internal koha auth number as authid during import, you need to change the bibliographic data accordingly before import - which will need programming.






--- Bug imported by chris@bigballofwax.co.nz 2010-05-21 01:24 UTC  ---

This bug was previously known as _bug_ 4226 at http://bugs.koha.org/cgi-bin/bugzilla3/show_bug.cgi?id=4226

Actual time not defined. Setting to 0.0
The original reporter of this bug does not have
   an account here. Reassigning to the person who moved
   it here: chris@bigballofwax.co.nz.
   Previous reporter was brian@jhu.edu.
Setting qa contact to the default for this product.
   This bug either had no qa contact or an invalid one.
CC member nicolas.morin@biblibre.com does not have an account here

Comment 1 Chris Cormack 2010-12-14 03:15:00 UTC
Anyone know what the current state of this is?
Comment 2 lculber 2011-05-02 14:42:02 UTC
Has this been superseded by bug 6113 that Galen is working on or is it otherwise related?
Comment 3 Stefano Bargioni 2011-12-02 10:00:36 UTC Comment hidden (obsolete)
Comment 4 Magnus Enger 2012-04-17 19:40:17 UTC
Looks like this should be "needs signoff".
Comment 5 Chris Cormack 2012-06-24 07:23:21 UTC
Attached script looks good but needs the copyright headers fixed. Also should we change the description of this bug?
Comment 6 Chris Cormack 2012-07-01 01:02:35 UTC
Patch is not in git format, fixing now
Comment 7 Chris Cormack 2012-07-01 01:07:01 UTC
Created attachment 10578 [details] [review]
Bug 4226 : New script to keep 001 numbers when importing
Comment 8 Chris Cormack 2012-07-01 01:07:37 UTC
Git formatted patch with fixed copyright headers.
Comment 9 Julian Maurice 2012-08-09 14:07:39 UTC
Script seems to work as expected, but a few remarks:

1. GetOptions wait for a string after -a and -b parameters, which are boolean flags according to POD

2. --commit option is not used

3. binmode(STDOUT, ":utf8") is doing something strange. date outputs:
jeudi 9 août 2012, 15:47:12 (UTC+0200)
if i remove binmode line, it looks ok

4. some printed strings have newline at end, some others don't. Some strings are printed only when $verbose is true while others are always printed (can't figure out why). Maybe you should use some wrapper sub like this:
sub print_log {
   print @_, "\n" if $verbose;
}

5. italian language (lines 126-127) should be translated.

6. More important: what happens if some ids (authid or biblionumber) are the same when resetting the primary key?

I tested this:
perl misc/migration_tools/sysno_keeper.pl -a 1 -v -t 009
having 2 records containing the same data in 009 field.

Output:
1. dropping primary key for auth_header.authid
2. filling in auth_header.authid... 3. rebuilding primary key for auth_header.authid
DBD::mysql::db do failed: ALTER TABLE causes auto_increment resequencing, resulting in duplicate entry '12345678' for key 'PRIMARY' at misc/migration_tools/sysno_keeper.pl line 104.

Is there a way to detect such errors and cancel previous modifications ?


Failed QA for 1 and 2. A follow-up for 6 would be great!
Comment 10 Mark Tompsett 2015-10-19 18:55:27 UTC
Created attachment 43611 [details] [review]
Bug 4226: Follow to address comment #9

It would be appreciated if testing of this script was redone to
confirm if all the points were handled properly.

The only point truly outstanding still is 6.
Comment 11 Katrin Fischer 2023-01-07 21:28:26 UTC
We had some behaviour changes that possibly affect this, like bug 18014. We nowadays enforce the 001 to be Koha's authid consistently.

I am going to close this as INVALID, please reopen if you disagree!