Bugzilla – Attachment 35911 Details for
Bug 13703
batchRebuildBiblioTables.pl should also update from database to MARC Record.
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Help
|
New Account
|
Log In
[x]
|
Forgot Password
Login:
[x]
[patch]
Bug 13703 - batchRebuildBiblioTables.pl should also update from database to MARC Record.
Bug-13703---batchRebuildBiblioTablespl-should-also.patch (text/plain), 9.68 KB, created by
Olli-Antti Kivilahti
on 2015-02-13 16:16:30 UTC
(
hide
)
Description:
Bug 13703 - batchRebuildBiblioTables.pl should also update from database to MARC Record.
Filename:
MIME Type:
Creator:
Olli-Antti Kivilahti
Created:
2015-02-13 16:16:30 UTC
Size:
9.68 KB
patch
obsolete
>From 10971595ceaf3c9bcf811b36b399284ecb63f48f Mon Sep 17 00:00:00 2001 >From: Olli-Antti Kivilahti <olli-antti.kivilahti@jns.fi> >Date: Thu, 12 Feb 2015 16:59:48 +0200 >Subject: [PATCH] Bug 13703 - batchRebuildBiblioTables.pl should also update > from database to MARC Record. > >When adding a new column to biblio or biblioitems and populating it with legacy data, >the MARC Record needs to be updated to match the values in DB, so they can be indexed for searching. > >Upgrading the batchRebuildBiblioTables.pl -script to make updates upstream (DB columns to MARC Record) >and downstream (MARC Record to DB columns), based on the "Koha to MARC mappings". > >Also improves the batchRebuildBiblioTables.pl -script's documentation and parameter handling. > >TEST PLAN: > >1. Add a new column to biblio or biblioitems-tables. >2. Map the column to "Koha to MARC mappings". >3. Define the mapped MARC Field as a new search index to Zebra and all the relevant hassle what > comes with defining a new search index. >4. Populate legacy data to the new DB column. >5. Run the batchRebuildBiblioTables.pl to update the MARC Records from the new DB column. >6. Reindex the whole DB. >7. Make searches from your new index. >8. Split hairs :) >--- > C4/Biblio.pm | 82 ++++++++++++++++++++++++++++++++++++++ > misc/batchRebuildBiblioTables.pl | 78 ++++++++++++++++++++++++++++++------ > 2 files changed, 147 insertions(+), 13 deletions(-) > >diff --git a/C4/Biblio.pm b/C4/Biblio.pm >index 120c3aa..d25f311 100644 >--- a/C4/Biblio.pm >+++ b/C4/Biblio.pm >@@ -2251,6 +2251,88 @@ sub TransformKohaToMarc { > return $record; > } > >+=head UpdateKohaToMarc >+ >+ C4::Biblio::UpdateKohaToMarc($bibliodata, $record); >+ || >+ C4::Biblio::UpdateKohaToMarc($biblionumberOrBibliodata); >+ || >+ C4::Biblio::UpdateKohaToMarc($biblionumber); >+ >+Overwrites all MARC Fields mapped to Koha biblio or biblioitems -tables with >+their values from DB. >+items-table columns are not updated to the MARC Record, because they are no longer stored there. >+ >+@PARAM1 MANDATORY >+ Integer, biblionumber of the target Biblio. This updates all DB coluns to the MARC record which takes a long time. >+ OR >+ Hash, with keys matching column names of biblio- and biblioitems-tables. biblionumber must be present. >+ Eg. {biblio.biblionumber => 1213, biblioitems.agerestriction => 'K18', biblioitems.publishercode => 'PKA'} >+ This way only the given columns are updated to the MARC record, possibly skipping unnecessary work. >+@PARAM2 OPTIONAL >+ MARC::Record-object, to have it's Koha mapped fields updated. >+@RETURNS String, Error code: >+ 'NOBIBLIODATA' : Couldn't find the biblio columns from the given $biblionumberOrBibliodata, or the biblionumber is missing. >+ 'NORECORD' : The given $record is not a MARC::Record-object >+=cut >+sub UpdateKohaToMarc { >+ my ($biblionumberOrBibliodata, $record) = @_; >+ >+ my $bibdata; >+ >+ unless (ref $biblionumberOrBibliodata) { #We have a SCALAR, eg. a biblionumber. >+ $bibdata = {}; >+ my @biblioitems = C4::Biblio::GetBiblioItemByBiblioNumber($biblionumberOrBibliodata); >+ my $biblioitem = $biblioitems[0]; >+ return 'NOBIBLIODATA' unless $biblioitem; >+ map {$bibdata->{"biblioitems.$_"} = $biblioitem->{$_}} keys($biblioitem); >+ my $biblio = C4::Biblio::GetBiblio($biblionumberOrBibliodata); >+ return 'NOBIBLIODATA' unless $biblio; >+ map {$bibdata->{"biblio.$_"} = $biblio->{$_}} keys(%$biblio); >+ $bibdata->{biblionumber} = $biblionumberOrBibliodata; >+ } >+ else { >+ $bibdata = $biblionumberOrBibliodata; >+ } >+ my $biblionumber = $bibdata->{'biblio.biblionumber'} || $bibdata->{'biblio.biblionumber'}; >+ return 'NOBIBLIODATA' if (not($bibdata) || not($biblionumber)); >+ unless (ref $record eq 'MARC::Record') { >+ $record = C4::Biblio::GetMarcBiblio( $bibdata->{biblionumber} ); >+ } >+ return 'NORECORD' unless $record; >+ my $frameworkcode = ($bibdata->{frameworkcode}) ? $bibdata->{frameworkcode} : C4::Biblio::GetFrameworkCode($bibdata->{biblionumber}); >+ >+ my $db_to_marc = C4::Context->marcfromkohafield; #Get the mapping rules. >+ my $db_columns_as_fields = {}; >+ while ( my ($name, $value) = each %$bibdata ) { >+ next unless $value; >+ next unless my $dtm = $db_to_marc->{''}->{$name}; >+ next unless ( scalar( @$dtm ) ); >+ my ($tag, $letter) = @$dtm; >+ $tag .= ''; >+ foreach my $value ( split(/\s?\|\s?/, $value, -1) ) { >+ next if $value eq ''; >+ $db_columns_as_fields->{$tag} //= {}; >+ $db_columns_as_fields->{$tag}->{$letter} = $value; >+ } >+ } >+ foreach my $fieldCode (sort keys %$db_columns_as_fields) { >+ my $subfields = $db_columns_as_fields->{ $fieldCode }; >+ >+ my @existingFields = $record->field($fieldCode); >+ if ($existingFields[0]) { >+ $existingFields[0]->update(each $subfields); >+ } >+ else { >+ my $newField = MARC::Field->new($fieldCode, '', '', each $subfields); >+ $record->insert_fields_ordered($newField); >+ } >+ } >+ >+ C4::Biblio::ModBiblio($record, $biblionumber, $frameworkcode); >+ return undef; >+} >+ > =head2 PrepHostMarcField > > $hostfield = PrepHostMarcField ( $hostbiblionumber,$hostitemnumber,$marcflavour ) >diff --git a/misc/batchRebuildBiblioTables.pl b/misc/batchRebuildBiblioTables.pl >index ae4f54e..9f88f02 100755 >--- a/misc/batchRebuildBiblioTables.pl >+++ b/misc/batchRebuildBiblioTables.pl >@@ -20,30 +20,66 @@ use Time::HiRes qw(gettimeofday); > > use Getopt::Long; > my ( $input_marc_file, $number) = ('', 0); >-my ($version, $confirm, $test_parameter); >+my ($help, $confirm, $test_parameter); >+my ($updateUpstream, $updateDownstream, $limitedColumns); > GetOptions( >- 'c' => \$confirm, >- 'h' => \$version, >- 't' => \$test_parameter, >+ 'c|confirm' => \$confirm, >+ 'h|help' => \$help, >+ 't|test' => \$test_parameter, >+ 'u|upstream' => \$updateUpstream, >+ 'd|downstream' => \$updateDownstream, >+ 'columns:s' => \$limitedColumns, > ); > >-if ($version || (!$confirm)) { >- print <<EOF >-This script rebuilds the non-MARC DB from the MARC values. >+my $helpText = <<EOF >+This script rebuilds the non-MARC DB from the MARC values and vice versa. > You can/must use it when you change your mapping. > > Example: you decide to map biblio.title to 200\$a (it was previously mapped to 610\$a). > Run this script or you will have strange results in OPAC ! > >+ -h | --help This friendly helper! >+ -u | --upstream Update MARC Records from the database columns using "Koha to MARC mappings". >+ -d | --downstream Update database columns from the MARC Records using "Koha to MARC mappings". >+ -c | --confirm Confirm, that you want to batch modify all of your Bibliographic records. >+ It is better to double check your mappings now since this operation cannot be undone. >+ This will take a long time... >+ -t | --test Test only, change nothing in DB >+ --columns NOT IMPLEMENTED YET! Limit the updates to this list of table >+ columns, instead of updating all "Koha to MARC mappings". >+ Example: --columns "biblioitems.publicationyear, biblioitems.datereceived, biblio.copyrightdate" >+ Only columns for tables biblio and biblioitems are supported. >+ > Syntax: >-\t./batchRebuildBiblioTables.pl -h (or without arguments => shows this screen) >-\t./batchRebuildBiblioTables.pl -c (c like confirm => rebuild non marc DB (may be long) >-\t-t => test only, change nothing in DB >+./batchRebuildBiblioTables.pl --help >+./batchRebuildBiblioTables.pl --test --upstream >+./batchRebuildBiblioTables.pl --confirm --downstream > EOF > ; >- exit; >+ >+if ($help) { >+ print $helpText; >+ exit(); >+} >+if (not($updateUpstream) && not($updateDownstream)) { >+ print $helpText."\n\nyou must choose either --upstream or --downstream !\n"; >+ exit(); >+} >+if (!$confirm) { >+ print $helpText."\n\nRead the help file!\n"; >+ exit(); >+} >+my @limitedColumns = split(/\s*,\s*/,$limitedColumns); >+if ($limitedColumns && scalar(@limitedColumns) == 0) { >+ print $helpText."\n\nCouldn't parse --columns '$limitedColumns'!\n"; >+ exit(); >+} >+elsif (scalar(@limitedColumns)) { >+ @limitedColumns = undef; >+ #TODO limited columns parsing. > } > >+ > my $dbh = C4::Context->dbh; > my $i=0; > my $starttime = time(); >@@ -56,6 +92,15 @@ my ($tagfield,$tagsubfield) = &GetMarcFromKohaField("items.itemnumber",''); > # $dbh->do("lock tables biblio write, biblioitems write, items write, marc_biblio write, marc_subfield_table write, marc_blob_subfield write, marc_word write, marc_subfield_structure write, stopwords write"); > my $sth = $dbh->prepare("SELECT biblionumber FROM biblio"); > $sth->execute; >+ >+if ($updateUpstream) { >+ while (my ($biblionumber) = $sth->fetchrow) { >+ DB_ToRecord($biblionumber, \@limitedColumns); >+ } >+ exit(); >+} >+ >+ > # my ($biblionumbermax) = $sth->fetchrow; > # warn "$biblionumbermax <<=="; > my @errors; >@@ -63,8 +108,8 @@ while (my ($biblionumber)= $sth->fetchrow) { > #now, parse the record, extract the item fields, and store them in somewhere else. > my $record = GetMarcBiblio($biblionumber); > if (not defined $record) { >- push @errors, $biblionumber; >- next; >+ push @errors, $biblionumber; >+ next; > } > my @fields = $record->field($tagfield); > my @items; >@@ -101,3 +146,10 @@ sub localNEWmodbiblio { > C4::Biblio::_koha_modify_biblioitem_nonmarc( $dbh, $oldbiblio ); > return 1; > } >+ >+sub DB_ToRecord { >+ my ($biblionumber) = @_; >+ >+ my $errors = C4::Biblio::UpdateKohaToMarc($biblionumber); >+ print "bn: $biblionumber, $errors\n" if $errors; >+} >-- >1.7.9.5
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Diff
|
Splinter Review
Attachments on
bug 13703
:
35877
| 35911