Bugzilla – Attachment 178168 Details for
Bug 29597
Add tomarcplugin option to bulkmarcimport.pl
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Help
|
New Account
|
Log In
[x]
|
Forgot Password
Login:
[x]
[patch]
Bug 29597: Add tomarcplugin option to bulkmarcimport.pl
Bug-29597-Add-tomarcplugin-option-to-bulkmarcimpor.patch (text/plain), 5.39 KB, created by
David Gustafsson
on 2025-02-18 10:39:38 UTC
(
hide
)
Description:
Bug 29597: Add tomarcplugin option to bulkmarcimport.pl
Filename:
MIME Type:
Creator:
David Gustafsson
Created:
2025-02-18 10:39:38 UTC
Size:
5.39 KB
patch
obsolete
>From 52495a70b446085180709d69d1b78449ae3fe0b5 Mon Sep 17 00:00:00 2001 >From: David Gustafsson <david.gustafsson@ub.gu.se> >Date: Tue, 18 Feb 2025 10:39:52 +0100 >Subject: [PATCH] Bug 29597: Add tomarcplugin option to bulkmarcimport.pl > >Add tomarcplugin option to bulkmarcimport script. > >To test: > >1) Place ToMarcExample.pm in a Koha plugin directory at Koha/Plugin/ToMarcExample.pm > and make sure its enabled and installed in Koha >2) Import some biblio with > ./bulkmarcimport.pl -v -b -file biblios.mrc -insert -update -c=MARC21 -n=1 -tomarcplugin="Koha::Plugin::ToMarcExample" >3) For the imported biblio, check that field 590a has been populated with > the value 'ToMarcExample added field' >--- > misc/migration_tools/bulkmarcimport.pl | 73 ++++++++++++++++++-------- > 1 file changed, 51 insertions(+), 22 deletions(-) > >diff --git a/misc/migration_tools/bulkmarcimport.pl b/misc/migration_tools/bulkmarcimport.pl >index 70721f05291..cc7aff21077 100755 >--- a/misc/migration_tools/bulkmarcimport.pl >+++ b/misc/migration_tools/bulkmarcimport.pl >@@ -42,6 +42,7 @@ use Koha::Logger; > use Koha::Biblios; > use Koha::SearchEngine; > use Koha::SearchEngine::Search; >+use Koha::Plugins::Handler; > > use open qw( :std :encoding(UTF-8) ); > binmode( STDOUT, ":encoding(UTF-8)" ); >@@ -76,6 +77,7 @@ my $framework = ''; > my $localcust; > my $marc_mod_template = ''; > my $marc_mod_template_id = -1; >++my $to_marc_plugin; > my $skip_indexing = 0; > my $skip_bad_records; > $| = 1; >@@ -114,6 +116,7 @@ GetOptions( > 'framework=s' => \$framework, > 'custom:s' => \$localcust, > 'marcmodtemplate:s' => \$marc_mod_template, >+ 'tomarcplugin:s' => \$to_marc_plugin, > 'si|skip_indexing' => \$skip_indexing, > 'sk|skip_bad_records' => \$skip_bad_records, > ); >@@ -262,29 +265,51 @@ my $searcher = Koha::SearchEngine::Search->new( > print "Characteristic MARC flavour: $marc_flavour\n" if $verbose; > my $starttime = gettimeofday; > >-# don't let MARC::Batch open the file, as it applies the ':utf8' IO layer >-my $fh = IO::File->new($input_marc_file) or die "Could not open $input_marc_file: $!"; >- >-if ( defined $format && $format =~ /XML/i ) { >- >- # ugly hack follows -- MARC::File::XML, when used by MARC::Batch, >- # appears to try to convert incoming XML records from MARC-8 >- # to UTF-8. Setting the BinaryEncoding key turns that off >- # TODO: see what happens to ISO-8859-1 XML files. >- # TODO: determine if MARC::Batch can be fixed to handle >- # XML records properly -- it probably should be >- # be using a proper push or pull XML parser to >- # extract the records, not using regexes to look >- # for <record>.*</record>. >- $MARC::File::XML::_load_args{BinaryEncoding} = 'utf-8'; >- my $recordformat = ( $marc_flavour eq "MARC21" ? "USMARC" : uc($marc_flavour) ); >- >- #UNIMARC Authorities have a different way to manage encoding than UNIMARC biblios. >- $recordformat = $recordformat . "AUTH" if ( $authorities and $marc_flavour ne "MARC21" ); >- $MARC::File::XML::_load_args{RecordFormat} = $recordformat; >- $batch = MARC::Batch->new( 'XML', $fh ); >+ >+if ($to_marc_plugin) { >+ my $marc_records_blob = undef; >+ { >+ local $/ = undef; >+ open FILE, $input_marc_file; >+ binmode FILE, ':raw'; >+ $marc_records_blob = <FILE>; >+ close FILE; >+ } >+ $marc_records_blob = Koha::Plugins::Handler->run( >+ { >+ class => $to_marc_plugin, >+ method => 'to_marc', >+ params => { data => $marc_records_blob } >+ } >+ ); >+ open(my $fh, '<', \$marc_records_blob); >+ binmode $fh, ':raw'; >+ $batch = MARC::Batch->new('USMARC', $fh); > } else { >- $batch = MARC::Batch->new( 'USMARC', $fh ); >+ # don't let MARC::Batch open the file, as it applies the ':utf8' IO layer >+ my $fh = IO::File->new($input_marc_file) or die "Could not open $input_marc_file: $!"; >+ >+ if ( defined $format && $format =~ /XML/i ) { >+ >+ # ugly hack follows -- MARC::File::XML, when used by MARC::Batch, >+ # appears to try to convert incoming XML records from MARC-8 >+ # to UTF-8. Setting the BinaryEncoding key turns that off >+ # TODO: see what happens to ISO-8859-1 XML files. >+ # TODO: determine if MARC::Batch can be fixed to handle >+ # XML records properly -- it probably should be >+ # be using a proper push or pull XML parser to >+ # extract the records, not using regexes to look >+ # for <record>.*</record>. >+ $MARC::File::XML::_load_args{BinaryEncoding} = 'utf-8'; >+ my $recordformat = ( $marc_flavour eq "MARC21" ? "USMARC" : uc($marc_flavour) ); >+ >+ #UNIMARC Authorities have a different way to manage encoding than UNIMARC biblios. >+ $recordformat = $recordformat . "AUTH" if ( $authorities and $marc_flavour ne "MARC21" ); >+ $MARC::File::XML::_load_args{RecordFormat} = $recordformat; >+ $batch = MARC::Batch->new( 'XML', $fh ); >+ } else { >+ $batch = MARC::Batch->new( 'USMARC', $fh ); >+ } > } > > $batch->warnings_off(); >@@ -1052,6 +1077,10 @@ If set, check the validity of records before adding. If they are invalid we will > print the output of MARC::Lint->check_record and skip them during the import. Without > this option bad records may kill the job. > >+=item B<-tomarcplugin>=I<PLUGIN> >+ >+Fully qualified class name of plugin with to_marc method for processing raw MARC data. >+ > =back > > =cut >-- >2.48.1
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Diff
|
Splinter Review
Attachments on
bug 29597
:
128181
|
128182
|
128361
| 178168