Bugzilla – Attachment 12152 Details for
Bug 2060
Ability to import Authorities from Staff Client
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Help
|
New Account
|
Log In
[x]
|
Forgot Password
Login:
[x]
[patch]
Bug 2060: Update command line MARC import scripts
Bug-2060-Update-command-line-MARC-import-scripts.patch (text/plain), 30.59 KB, created by
Jared Camins-Esakov
on 2012-09-11 19:03:14 UTC
(
hide
)
Description:
Bug 2060: Update command line MARC import scripts
Filename:
MIME Type:
Creator:
Jared Camins-Esakov
Created:
2012-09-11 19:03:14 UTC
Size:
30.59 KB
patch
obsolete
>From f1895fff7ea10ed366c313a70ae0bb45840852df Mon Sep 17 00:00:00 2001 >From: Jared Camins-Esakov <jcamins@cpbibliography.com> >Date: Mon, 25 Jun 2012 11:38:19 -0400 >Subject: [PATCH] Bug 2060: Update command line MARC import scripts > >Expose authority import functionality to the command line import >scripts, and rename them from commit_biblios_file.pl and >stage_biblios_file.pl to commit_file.pl and stage_file.pl. > >To test (note that these instructions assume you have a MARC21 >installation and are using the provided sample file): >1. Find a file of authorities (a sample file with MARC21 authorities > is attached to bug 7475) and download it to your server >2. Stage the file using the following command (replace <filename> with > the name of the file you saved in step 1): > > misc/stage_file.pl --file <filename> --authorities >3. Note the batch number the script assigns to your batch >4. Commit the records using the following command (replace <batchnumber> > with the batch number you made note of in step 3): > > misc/commit_file.pl --batch-number <batchnumber> >5. Index the authorities Zebraqueue (or wait) >6. Confirm that the new authorities appear. >7. Create a matching rule with the following settings: > Code: AUTHTEST > Description: Personal name main entry > Match threshold: 999 > Record type: Authority record > Search index: Heading-main > Score: 1000 > Tag: 100 > Subfields: a > Offset: 0 > Length: 0 > (note the ID of this matching rule) >8. Stage the authority file again, this time using the following > command: > > misc/stage_file.pl --file <filename> --authorities \ > --match <matchingrule> >7. Revert the import with the following command: > > misc/commit_file.pl --batch-number <batchnumber> --revert >8. Index the authorities Zebraqueue (or wait) >9. Confirm that the records have been removed >10. Import an authority record with the Stage MARC/Manage staged MARC > tools in exactly the way you would for a bibliographic record, > but choose "Authority" instead of "Bibliographic" for the record > type. > >Signed-off-by: Elliott Davis <elliott@bywatersolutions.com> > >Testing plan delivers as it should. > >Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com> >Rebased on latest master 11 September 2012 >--- > C4/AuthoritiesMarc.pm | 22 +++--- > C4/Context.pm | 2 +- > C4/ImportBatch.pm | 16 ++-- > installer/data/mysql/kohastructure.sql | 2 +- > .../prog/en/modules/tools/manage-marc-import.tt | 53 ++++++++---- > .../prog/en/modules/tools/stage-marc-import.tt | 28 ++++++- > misc/{commit_biblios_file.pl => commit_file.pl} | 56 ++++++++++--- > misc/{stage_biblios_file.pl => stage_file.pl} | 84 +++++++++++--------- > tools/manage-marc-import.pl | 1 + > tools/stage-marc-import.pl | 3 +- > 10 files changed, 176 insertions(+), 91 deletions(-) > rename misc/{commit_biblios_file.pl => commit_file.pl} (62%) > rename misc/{stage_biblios_file.pl => stage_file.pl} (67%) > >diff --git a/C4/AuthoritiesMarc.pm b/C4/AuthoritiesMarc.pm >index e026fe1..ac65055 100644 >--- a/C4/AuthoritiesMarc.pm >+++ b/C4/AuthoritiesMarc.pm >@@ -206,16 +206,18 @@ sub SearchAuthorities { > # also store main entry MARC tag, to extract it at end of search > my $mainentrytag; > ##first set the authtype search and may be multiple authorities >- my $n=0; >- my @authtypecode; >- my @auths=split / /,$authtypecode ; >- foreach my $auth (@auths){ >- $query .=" \@attr 1=authtype \@attr 5=100 ".$auth; ##No truncation on authtype >- push @authtypecode ,$auth; >- $n++; >- } >- if ($n>1){ >- while ($n>1){$query= "\@or ".$query;$n--;} >+ if ($authtypecode) { >+ my $n=0; >+ my @authtypecode; >+ my @auths=split / /,$authtypecode ; >+ foreach my $auth (@auths){ >+ $query .=" \@attr 1=authtype \@attr 5=100 ".$auth; ##No truncation on authtype >+ push @authtypecode ,$auth; >+ $n++; >+ } >+ if ($n>1){ >+ while ($n>1){$query= "\@or ".$query;$n--;} >+ } > } > > my $dosearch; >diff --git a/C4/Context.pm b/C4/Context.pm >index 0a56aa8..9748f72 100644 >--- a/C4/Context.pm >+++ b/C4/Context.pm >@@ -989,7 +989,7 @@ set_userenv is called in Auth.pm > #' > sub set_userenv { > my ($usernum, $userid, $usercnum, $userfirstname, $usersurname, $userbranch, $branchname, $userflags, $emailaddress, $branchprinter)= @_; >- my $var=$context->{"activeuser"}; >+ my $var=$context->{"activeuser"} || ''; > my $cell = { > "number" => $usernum, > "id" => $userid, >diff --git a/C4/ImportBatch.pm b/C4/ImportBatch.pm >index 5149f9b..50b10cf 100644 >--- a/C4/ImportBatch.pm >+++ b/C4/ImportBatch.pm >@@ -205,7 +205,7 @@ sub AddImportBatch { > my (@fields, @vals); > foreach (qw( matcher_id template_id branchcode > overlay_action nomatch_action item_action >- import_status batch_type file_name comments )) { >+ import_status batch_type file_name comments record_type )) { > if (exists $params->{$_}) { > push @fields, $_; > push @vals, $params->{$_}; >@@ -328,7 +328,7 @@ sub BatchStageMarcRecords { > my $branch_code = shift; > my $parse_items = shift; > my $leave_as_staging = shift; >- >+ > # optional callback to monitor status > # of job > my $progress_interval = 0; >@@ -346,6 +346,7 @@ sub BatchStageMarcRecords { > batch_type => 'batch', > file_name => $file_name, > comments => $comments, >+ record_type => $record_type, > } ); > if ($parse_items) { > SetImportBatchItemAction($batch_id, 'always_add'); >@@ -584,7 +585,7 @@ sub BatchCommitRecords { > $num_items_errored += $bib_items_errored; > } > } else { >- my $authid = AddAuthority($marc_record, undef, GuessAuthTypeCode($marc_record)); >+ $recordid = AddAuthority($marc_record, undef, GuessAuthTypeCode($marc_record)); > $query = "UPDATE import_auths SET matched_authid = ? WHERE import_record_id = ?"; > } > my $sth = $dbh->prepare_cached($query); >@@ -616,7 +617,7 @@ sub BatchCommitRecords { > $num_items_errored += $bib_items_errored; > } > } else { >- my $oldxml = GetAuthorityXML($recordid); >+ $oldxml = GetAuthorityXML($recordid); > > ModAuthority($recordid, $marc_record, GuessAuthTypeCode($marc_record)); > $query = "UPDATE import_auths SET matched_authid = ? WHERE import_record_id = ?"; >@@ -1350,9 +1351,9 @@ sub _add_auth_fields { > if ($marc_record->field('001')) { > $controlnumber = $marc_record->field('001')->data(); > } >- my $authorized_heading = GetAuthorizedHeading($marc_record); >+ my $authorized_heading = GetAuthorizedHeading({ record => $marc_record }); > my $dbh = C4::Context->dbh; >- my $sth = $dbh->prepare("INSERT INTO import_auths (import_record_id, controlnumber, authorized_heading) VALUES (?, ?, ?)"); >+ my $sth = $dbh->prepare("INSERT INTO import_auths (import_record_id, control_number, authorized_heading) VALUES (?, ?, ?)"); > $sth->execute($import_record_id, $controlnumber, $authorized_heading); > $sth->finish(); > } >@@ -1403,8 +1404,7 @@ sub _update_batch_record_counts { > num_records = ( > SELECT COUNT(*) > FROM import_records >- WHERE import_batch_id = import_batches.import_batch_id >- AND record_type = 'biblio'), >+ WHERE import_batch_id = import_batches.import_batch_id), > num_items = ( > SELECT COUNT(*) > FROM import_records >diff --git a/installer/data/mysql/kohastructure.sql b/installer/data/mysql/kohastructure.sql >index f02929c..7d18b60 100644 >--- a/installer/data/mysql/kohastructure.sql >+++ b/installer/data/mysql/kohastructure.sql >@@ -924,7 +924,7 @@ CREATE TABLE `import_auths` ( > `original_source` varchar(25) default NULL, > CONSTRAINT `import_auths_ibfk_1` FOREIGN KEY (`import_record_id`) > REFERENCES `import_records` (`import_record_id`) ON DELETE CASCADE ON UPDATE CASCADE, >- KEY `matched_authid` (`matched_authid`), >+ KEY `matched_authid` (`matched_authid`) > ) ENGINE=InnoDB DEFAULT CHARSET=utf8; > > -- >diff --git a/koha-tmpl/intranet-tmpl/prog/en/modules/tools/manage-marc-import.tt b/koha-tmpl/intranet-tmpl/prog/en/modules/tools/manage-marc-import.tt >index 2a9deda..8265b50 100644 >--- a/koha-tmpl/intranet-tmpl/prog/en/modules/tools/manage-marc-import.tt >+++ b/koha-tmpl/intranet-tmpl/prog/en/modules/tools/manage-marc-import.tt >@@ -1,3 +1,22 @@ >+[% BLOCK final_match_link %] >+ [% IF ( record.record_type == 'biblio' ) %] >+ <a href="/cgi-bin/koha/catalogue/detail.pl?biblionumber=[% record.final_match_id %]">[% record.final_match_id %]</a> >+ [% ELSIF ( record.record_type == 'auth' ) %] >+ <a href="/cgi-bin/koha/authorities/detail.pl?authid=[% record.final_match_id %]">[% record.final_match_id %]</a> >+ [% END %] >+[% END %] >+[% BLOCK match_link %] >+ [% IF ( record_lis.match_id ) %] >+ <tr> >+ <td /> >+ [% IF ( record.record_type == 'biblio' ) %] >+ <td class="highlight" colspan="4">Matches biblio [% record_lis.match_id %] (score = [% record_lis.match_score %]): <a href="/cgi-bin/koha/catalogue/detail.pl?biblionumber=[% record_lis.match_id %]">[% record_lis.match_citation %]</a></td> >+ [% ELSIF ( record.record_type == 'auth' ) %] >+ <td class="highlight" colspan="4">Matches authority [% record_lis.match_id %] (score = [% record_lis.match_score %]): <a href="/cgi-bin/koha/authorities/detail.pl?authid=[% record_lis.match_id %]">[% record_lis.match_citation %]</a></td> >+ [% END %] >+ </tr> >+ [% END %] >+[% END %] > [% INCLUDE 'doc-head-open.inc' %] > <title>Koha › Tools › Manage staged MARC records > [% IF ( import_batch_id ) %] >@@ -107,7 +126,8 @@ $(document).ready(function(){ > <fieldset class="rows" id="staged-record-matching-rules"> > <ol> > <li><span class="label">File name:</span> [% file_name %]</li> >- <li><span class="label">Comments:</span> [% IF ( comments ) %][% comments %](none)[% END %]</li> >+ <li><span class="label">Comments:</span> [% IF ( comments ) %][% comments %][% ELSE %](none)[% END %]</li> >+ <li><span class="label">Type:</span> [% IF ( record_type == 'auth' ) %]Authority records[% ELSE %]Bibliographic records[% END %]</li> > <li><span class="label">Staged:</span> [% upload_timestamp %]</li> > <li><span class="label">Status:</span> [% import_status %]</li> > <li> >@@ -218,9 +238,10 @@ Page > <th>#</th> > <th>File name</th> > <th>Comments</th> >+ <th>Type</th> > <th>Status</th> > <th>Staged</th> >- <th># Bibs</th> >+ <th># Records</th> > <th># Items</th> > <th>Action</th> > </tr> >@@ -229,9 +250,10 @@ Page > <td>[% batch_lis.import_batch_id %]</td> > <td><a href="[% batch_lis.script_name %]?import_batch_id=[% batch_lis.import_batch_id %]">[% batch_lis.file_name %]</a></td> > <td>[% batch_lis.comments %]</td> >+ <td>[% IF ( batch_lis.record_type == 'auth' ) %]Authority[% ELSE %]Bibliographic[% END %]</td> > <td>[% batch_lis.import_status %]</td> > <td>[% batch_lis.upload_timestamp %]</td> >- <td>[% batch_lis.num_biblios %]</td> >+ <td>[% batch_lis.num_records %]</td> > <td>[% batch_lis.num_items %][% IF ( batch_lis.num_items ) %] <a href="[% batch_lis.script_name %]?import_batch_id=[% batch_lis.import_batch_id %]&op=create_labels">(Create label batch)</a>[% END %]</td> > <td>[% IF ( batch_lis.can_clean ) %] > <form method="post" action="[% batch_lis.script_name %]" name="clean_batch_[% batch_lis.import_batch_id %]" id="clean_batch_[% batch_lis.import_batch_id %]" > >@@ -258,7 +280,7 @@ Page > [% END %] > [% END %] > >-[% IF ( biblio_list ) %] >+[% IF ( record_list ) %] > [% IF ( pages ) %] > <div class="pages"> > Page >@@ -277,26 +299,21 @@ Page > <th>Citation</th> > <th>Status</th> > <th>Match?</th> >- <th>Bib</th> >+ <th>Record</th> > > </tr> >- [% FOREACH biblio_lis IN biblio_list %] >+ [% FOREACH record_lis IN record_list %] > [% UNLESS ( loop.odd ) %]<tr class="highlight">[% ELSE %]<tr>[% END %] >- <td>[% biblio_lis.record_sequence %]</td> >- <td><a href="/cgi-bin/koha/catalogue/showmarc.pl?importid=[% biblio_lis.import_record_id %]" rel="gb_page_center[600,500]">[% biblio_lis.citation %]</a></td> >- <td>[% biblio_lis.status %]</td> >- <td>[% biblio_lis.overlay_status %]</td> >- <td>[% IF ( biblio_lis.final_match_biblionumber ) %] >- <a href="/cgi-bin/koha/catalogue/detail.pl?biblionumber=[% biblio_lis.final_match_biblionumber %]">[% biblio_lis.final_match_biblionumber %]</a> >+ <td>[% record_lis.record_sequence %]</td> >+ <td><a href="/cgi-bin/koha/catalogue/showmarc.pl?importid=[% record_lis.import_record_id %]" rel="gb_page_center[600,500]">[% record_lis.citation %]</a></td> >+ <td>[% record_lis.status %]</td> >+ <td>[% record_lis.overlay_status %]</td> >+ <td>[% IF ( record_lis.final_match_id ) %] >+ [% PROCESS final_match_link record=record_lis %] > [% END %] > </td> > </tr> >- [% IF ( biblio_lis.match_biblionumber ) %] >- <tr> >- <td /> >- <td class="highlight" colspan="4">Matches biblio [% biblio_lis.match_biblionumber %] (score = [% biblio_lis.match_score %]): <a href="/cgi-bin/koha/catalogue/detail.pl?biblionumber=[% biblio_lis.match_biblionumber %]">[% biblio_lis.match_citation %]</a></td> >- </tr> >- [% END %] >+ [% PROCESS match_link record=record_lis %] > [% END %] > </table> > [% IF ( pages ) %] >diff --git a/koha-tmpl/intranet-tmpl/prog/en/modules/tools/stage-marc-import.tt b/koha-tmpl/intranet-tmpl/prog/en/modules/tools/stage-marc-import.tt >index 478527c..4846872 100644 >--- a/koha-tmpl/intranet-tmpl/prog/en/modules/tools/stage-marc-import.tt >+++ b/koha-tmpl/intranet-tmpl/prog/en/modules/tools/stage-marc-import.tt >@@ -11,6 +11,13 @@ > //<![CDATA[ > $(document).ready(function(){ > $("#processfile").hide(); >+ $("#record_type").change(function() { >+ if ($(this).val() == 'auth') { >+ $('#items').hide(); >+ } else { >+ $('#items').show(); >+ } >+ }); > }); > function CheckForm(f) { > if ($("#fileToUpload").value == '') { >@@ -40,6 +47,12 @@ function CheckForm(f) { > [% IF ( uploadmarc ) %] > <p>MARC staging results :</p> > <ul> >+ [% SWITCH (record_type) %] >+ [% CASE 'biblio' %] >+ <li>Processing bibliographic records</li> >+ [% CASE 'auth' %] >+ <li>Processing authority records</li> >+ [% END %] > <li>[% total %] records in file</li> > <li>[% import_errors %] records not staged because of MARC error</li> > <li>[% staged %] records staged</li> >@@ -62,8 +75,8 @@ function CheckForm(f) { > </ul> > [% ELSE %] > <ul> >- <li>Select a MARC file to stage in the import reservoir. It will be parsed, and each valid record staged for later import into the catalog.</li> >- <li>You can enter a name for this import. It may be useful, when creating a biblio, to remember where the suggested MARC data comes from!</li> >+ <li>Select a MARC file to stage in the import reservoir. It will be parsed, and each valid record staged for later import into the catalog.</li> >+ <li>You can enter a name for this import. It may be useful, when creating a record, to remember where the suggested MARC data comes from!</li> > </ul> > <form method="post" action="[% SCRIPT_NAME %]" id="uploadfile" enctype="multipart/form-data"> > <fieldset class="rows" id="uploadform"> >@@ -92,6 +105,13 @@ function CheckForm(f) { > <input type="text" id="comments" name="comments" /> > > </li> >+ <li> >+ <label for='record_type'>Record type:</label> >+ <select name='record_type' id='record_type'> >+ <option value='biblio' selected='selected'>Bibliographic</option> >+ <option value='auth'>Authority</option> >+ </select> >+ </li> > <li> > <label for="encoding">Character encoding: </label> > <select name="encoding" id="encoding"><option value="utf8" selected="selected">UTF-8 (Default)</option><option value="MARC-8">MARC 8</option><option value="ISO_5426">ISO 5426</option><option value="ISO_6937">ISO 6937</option><option value=ISO_8859-1">ISO 8859-1</option><option value="EUC-KR">EUC-KR</option></select> >@@ -108,7 +128,7 @@ function CheckForm(f) { > [% END %] > </select> > </li> >- <li><label for="overlay_action">Action if matching bibliographic record found: </label> >+ <li><label for="overlay_action">Action if matching record found: </label> > [% INCLUDE 'tools-overlay-action.inc' %] > </li> > <li><label for="nomatch_action">Action if no match is found: </label> >@@ -116,7 +136,7 @@ function CheckForm(f) { > </li> > </ol> > </fieldset> >- <fieldset class="rows"> >+ <fieldset class="rows" id="items"> > <legend>Check for embedded item record data?</legend> > <ol> > <li class="radio"> >diff --git a/misc/commit_biblios_file.pl b/misc/commit_file.pl >similarity index 62% >rename from misc/commit_biblios_file.pl >rename to misc/commit_file.pl >index b4be670..9c1089a 100755 >--- a/misc/commit_biblios_file.pl >+++ b/misc/commit_file.pl >@@ -1,7 +1,7 @@ > #!/usr/bin/perl > > use strict; >-#use warnings; FIXME - Bug 2505 >+use warnings; > BEGIN { > # find Koha's Perl modules > # test carefully before changing this >@@ -18,11 +18,13 @@ $| = 1; > # command-line parameters > my $batch_number = ""; > my $list_batches = 0; >+my $revert = 0; > my $want_help = 0; > > my $result = GetOptions( > 'batch-number:s' => \$batch_number, > 'list-batches' => \$list_batches, >+ 'revert' => \$revert, > 'h|help' => \$want_help > ); > >@@ -45,9 +47,15 @@ $dbh->{AutoCommit} = 0; > if ($batch_number =~ /^\d+$/ and $batch_number > 0) { > my $batch = GetImportBatch($batch_number); > die "$0: import batch $batch_number does not exist in database\n" unless defined $batch; >- die "$0: import batch $batch_number status is '" . $batch->{'import_status'} . "', and therefore cannot be imported\n" >- unless $batch->{'import_status'} eq "staged" or $batch->{'import_status'} eq "reverted"; >- process_batch($batch_number); >+ if ($revert) { >+ die "$0: import batch $batch_number status is '" . $batch->{'import_status'} . "', and therefore cannot be imported\n" >+ unless $batch->{'import_status'} eq "imported"; >+ revert_batch($batch_number); >+ } else { >+ die "$0: import batch $batch_number status is '" . $batch->{'import_status'} . "', and therefore cannot be imported\n" >+ unless $batch->{'import_status'} eq "staged" or $batch->{'import_status'} eq "reverted"; >+ process_batch($batch_number); >+ } > $dbh->commit(); > } else { > die "$0: please specify a numeric batch ID\n"; >@@ -74,8 +82,8 @@ sub process_batch { > my ($import_batch_id) = @_; > > print "... importing MARC records -- please wait\n"; >- my ($num_added, $num_updated, $num_items_added, $num_items_errored, $num_ignored) = >- BatchCommitBibRecords($import_batch_id, '', 100, \&print_progress_and_commit); >+ my ($num_added, $num_updated, $num_items_added, $num_items_errored, $num_ignored) = >+ BatchCommitRecords($import_batch_id, '', 100, \&print_progress_and_commit); > print "... finished importing MARC records\n"; > > print <<_SUMMARY_; >@@ -83,17 +91,40 @@ sub process_batch { > MARC record import report > ---------------------------------------- > Batch number: $import_batch_id >-Number of new bibs added: $num_added >-Number of bibs replaced: $num_updated >-Number of bibs ignored: $num_ignored >+Number of new records added: $num_added >+Number of records replaced: $num_updated >+Number of records ignored: $num_ignored > Number of items added: $num_items_added > Number of items ignored: $num_items_errored > >-Note: an item is ignored if its barcode is a >+Note: an item is ignored if its barcode is a > duplicate of one already in the database. > _SUMMARY_ > } > >+sub revert_batch { >+ my ($import_batch_id) = @_; >+ >+ print "... reverting batch -- please wait\n"; >+ my ($num_deleted, $num_errors, $num_reverted, $num_items_deleted, $num_ignored) = >+ BatchRevertRecords($import_batch_id, 100, \&print_progress_and_commit); >+ print "... finished reverting batch\n"; >+ >+ print <<_SUMMARY_; >+ >+MARC record import report >+---------------------------------------- >+Batch number: $import_batch_id >+Number of records deleted: $num_deleted >+Number of errors: $num_errors >+Number of records reverted: $num_reverted >+Number of records ignored: $num_ignored >+Number of items added: $num_items_deleted >+ >+_SUMMARY_ >+} >+ >+ > sub print_progress_and_commit { > my $recs = shift; > print "... processed $recs records\n"; >@@ -106,7 +137,7 @@ $0: import a batch of staged MARC records into database. > > Use this batch job to complete the import of a batch of > MARC records that was staged either by the batch job >-stage_biblios_file.pl or by the Koha Tools option >+stage_file.pl or by the Koha Tools option > "Stage MARC Records for Import". > > Parameters: >@@ -114,6 +145,7 @@ Parameters: > to import > --list-batches print a list of record batches > available to commit >- --help or -h show this message. >+ --revert revert a batch instead of importing it >+ --help or -h show this message. > _USAGE_ > } >diff --git a/misc/stage_biblios_file.pl b/misc/stage_file.pl >similarity index 67% >rename from misc/stage_biblios_file.pl >rename to misc/stage_file.pl >index 6e2b30c..4ce16b5 100755 >--- a/misc/stage_biblios_file.pl >+++ b/misc/stage_file.pl >@@ -4,6 +4,7 @@ > # > # Copyright (C) 2007 LibLime > # Parts Copyright BSZ 2011 >+# Parts Copyright C & P Bibliography Services 2012 > # > # Koha is free software; you can redistribute it and/or modify it under the > # terms of the GNU General Public License as published by the Free Software >@@ -35,8 +36,10 @@ use Getopt::Long; > $| = 1; > > # command-line parameters >+my $record_type = "biblio"; > my $encoding = ""; >-my $match_bibs = 0; >+my $authorities = 0; >+my $match = 0; > my $add_items = 0; > my $input_file = ""; > my $batch_comment = ""; >@@ -46,13 +49,16 @@ my $no_replace ; > my $result = GetOptions( > 'encoding:s' => \$encoding, > 'file:s' => \$input_file, >- 'match-bibs:s' => \$match_bibs, >+ 'match|match-bibs:s' => \$match, > 'add-items' => \$add_items, > 'no-replace' => \$no_replace, > 'comment:s' => \$batch_comment, >+ 'authorities' => \$authorities, > 'h|help' => \$want_help > ); > >+$record_type = 'auth' if ($authorities); >+ > if ($encoding eq "") { > $encoding = "utf8"; > } >@@ -68,13 +74,13 @@ unless (-r $input_file) { > > my $dbh = C4::Context->dbh; > $dbh->{AutoCommit} = 0; >-process_batch($input_file, $match_bibs, $add_items, $batch_comment); >+process_batch($input_file, $record_type, $match, $add_items, $batch_comment); > $dbh->commit(); > > exit 0; > > sub process_batch { >- my ($input_file, $match_bibs, $add_items, $batch_comment) = @_; >+ my ($input_file, $record_type, $match, $add_items, $batch_comment) = @_; > > open IN, "<$input_file" or die "$0: cannot open input file $input_file: $!\n"; > my $marc_records = ""; >@@ -92,50 +98,53 @@ sub process_batch { > close IN; > > print "... staging MARC records -- please wait\n"; >- my ($batch_id, $num_valid, $num_items, @import_errors) = >- BatchStageMarcRecords($encoding, $marc_records, $input_file, $batch_comment, '', $add_items, 0, >+ my ($batch_id, $num_valid_records, $num_items, @import_errors) = >+ BatchStageMarcRecords($record_type, $encoding, $marc_records, $input_file, $batch_comment, '', $add_items, 0, > 100, \&print_progress_and_commit); > print "... finished staging MARC records\n"; > > my $num_with_matches = 0; >- if ($match_bibs) { >- my $matcher = C4::Matcher->fetch($match_bibs) ; >- if (! defined $matcher) { >- $matcher = C4::Matcher->new('biblio'); >+ if ($match) { >+ my $matcher = C4::Matcher->fetch($match) ; >+ if (defined $matcher) { >+ SetImportBatchMatcher($batch_id, $match); >+ } elsif ($record_type eq 'biblio') { >+ $matcher = C4::Matcher->new($record_type); > $matcher->add_simple_matchpoint('isbn', 1000, '020', 'a', -1, 0, ''); >- $matcher->add_simple_required_check('245', 'a', -1, 0, '', >+ $matcher->add_simple_required_check('245', 'a', -1, 0, '', > '245', 'a', -1, 0, ''); >- } else { >- SetImportBatchMatcher($batch_id, $match_bibs); > } > # set default record overlay behavior > SetImportBatchOverlayAction($batch_id, ($no_replace) ? 'ignore' : 'replace'); > SetImportBatchNoMatchAction($batch_id, 'create_new'); > SetImportBatchItemAction($batch_id, 'always_add'); > print "... looking for matches with records already in database\n"; >- $num_with_matches = BatchFindBibDuplicates($batch_id, $matcher, 10, 100, \&print_progress_and_commit); >+ $num_with_matches = BatchFindDuplicates($batch_id, $matcher, 10, 100, \&print_progress_and_commit); > print "... finished looking for matches\n"; > } > >- my $num_invalid_bibs = scalar(@import_errors); >+ my $num_invalid_records = scalar(@import_errors); > print <<_SUMMARY_; > > MARC record staging report > ------------------------------------ >-Input file: $input_file >-Number of input bibs: $num_input_records >-Number of valid bibs: $num_valid >-Number of invalid bibs: $num_invalid_bibs >+Input file: $input_file >+Record type: $record_type >+Number of input records: $num_input_records >+Number of valid records: $num_valid_records >+Number of invalid records: $num_invalid_records > _SUMMARY_ >- if ($match_bibs) { >- print "Number of bibs matched: $num_with_matches\n"; >+ if ($match) { >+ print "Number of records matched: $num_with_matches\n"; > } else { >- print "Incoming bibs not matched against existing bibs (--match-bibs option not supplied)\n"; >+ print "Incoming records not matched against existing records (--match option not supplied)\n"; > } >- if ($add_items) { >- print "Number of items parsed: $num_items\n"; >- } else { >- print "No items parsed (--add-items option not supplied)\n"; >+ if ($record_type eq 'biblio') { >+ if ($add_items) { >+ print "Number of items parsed: $num_items\n"; >+ } else { >+ print "No items parsed (--add-items option not supplied)\n"; >+ } > } > > print "\n"; >@@ -151,31 +160,34 @@ sub print_progress_and_commit { > > sub print_usage { > print <<_USAGE_; >-$0: stage MARC bib file into reservoir. >+$0: stage MARC file into reservoir. > >-Use this batch job to load a file of MARC bibliographic records >-(with optional item information) into the Koha reservoir. >+Use this batch job to load a file of MARC bibliographic >+(with optional item information) or authority records into >+the Koha reservoir. > > After running this program to stage your file, you can use >-either the batch job commit_biblios_file.pl or the Koha >+either the batch job commit_file.pl or the Koha > Tools option "Manage Staged MARC Records" to load the > records into the main Koha database. > > Parameters: > --file <file_name> name of input MARC bib file >+ --authorities stage authority records instead of bibs > --encoding <encoding> encoding of MARC records, default is utf8. > Other possible options are: MARC-8, > ISO_5426, ISO_6937, ISO_8859-1, EUC-KR >- --match-bibs <match_id> use this option to match bibs >- in the file with bibs already in >+ --match <match_id> use this option to match records >+ in the file with records already in > the database for future overlay. >- If <match_id> isn't defined, a default >- MARC21 ISBN & title match rule will be applied. >+ If <match_id> isn't defined, a default >+ MARC21 ISBN & title match rule will be applied >+ for bib imports. > --add-items use this option to specify that > item data is embedded in the MARC > bibs and should be parsed. >- --no-replace overlay action for bib record: default is to >- replace extant bib with the imported record. >+ --no-replace overlay action for record: default is to >+ replace extant with the imported record. > --comment <comment> optional comment to describe > the record batch; if the comment > has spaces in it, surround the >diff --git a/tools/manage-marc-import.pl b/tools/manage-marc-import.pl >index de2cbf6..6f78d50 100755 >--- a/tools/manage-marc-import.pl >+++ b/tools/manage-marc-import.pl >@@ -29,6 +29,7 @@ use MARC::File::USMARC; > use C4::Context; > use C4::Koha; > use C4::Auth; >+use C4::AuthoritiesMarc; > use C4::Output; > use C4::Biblio; > use C4::ImportBatch; >diff --git a/tools/stage-marc-import.pl b/tools/stage-marc-import.pl >index ef3ade6..fbb5365 100755 >--- a/tools/stage-marc-import.pl >+++ b/tools/stage-marc-import.pl >@@ -55,6 +55,7 @@ my $nomatch_action = $input->param('nomatch_action'); > my $parse_items = $input->param('parse_items'); > my $item_action = $input->param('item_action'); > my $comments = $input->param('comments'); >+my $record_type = $input->param('record_type'); > my $encoding = $input->param('encoding'); > my ($template, $loggedinuser, $cookie) > = get_template_and_user({template_name => "tools/stage-marc-import.tmpl", >@@ -115,7 +116,7 @@ if ($completedJobID) { > # close STDOUT to signal to Apache that > # we're now running in the background > close STDOUT; >- close STDERR; >+ # close STDERR; # there is no good reason to close STDERR > } else { > # fork failed, so exit immediately > warn "fork failed while attempting to run $ENV{'SCRIPT_NAME'} as a background job"; >-- >1.7.2.5
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Diff
|
Splinter Review
Attachments on
bug 2060
:
10893
|
10894
|
10895
|
10899
|
10900
|
10901
|
10902
|
11258
|
11259
|
11260
|
12150
|
12151
|
12152
|
12195
|
12198
|
12213
|
12215
|
12218
|
12346
|
12347
|
12348