Bugzilla – Attachment 37918 Details for
Bug 11300
Add a new authority linker which searches for authority links on a Z39.50 server.
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Help
|
New Account
|
Log In
[x]
|
Forgot Password
Login:
[x]
[patch]
Bug 11300 - Add a Z39.50 Linker that searches for authority links in the specified Z39.50 server
Bug-11300---Add-a-Z3950-Linker-that-searches-for-a.patch (text/plain), 20.97 KB, created by
Blou
on 2015-04-15 20:32:21 UTC
(
hide
)
Description:
Bug 11300 - Add a Z39.50 Linker that searches for authority links in the specified Z39.50 server
Filename:
MIME Type:
Creator:
Blou
Created:
2015-04-15 20:32:21 UTC
Size:
20.97 KB
patch
obsolete
>From 8f4732dd2e716a98f188753422b24f0fb61d1d9e Mon Sep 17 00:00:00 2001 >From: Blou <philippe.blouin@inlibro.com> >Date: Wed, 15 Apr 2015 16:30:46 -0400 >Subject: [PATCH] Bug 11300 - Add a Z39.50 Linker that searches for authority > links in the specified Z39.50 server > >This patch depends on code added in Bug 11299, and must be applied after it. > >Sponsored-by: CCSR ( http://www.ccsr.qc.ca ) > >This patch adds a Z39.50 Linker which searches for authority links on a remote server. > >If a matching authority is found, it's imported in the local database for linking with the current biblio record. > >If no matching authority was found on the remote server, the Linker falls back to a local authority search. > >Configuration : >* The option "Z39.50 Server" is added to the LinkerModule preference. You must choose this to use the new Linker. >* The preference LinkerZ3950Server is required and specifies which server to use for linking. It must contain the "name" of the server, as defined in the z3950servers table. >* You can set the "local_first" option in the LinkerOptions preference to force the Linker to search for authorities in the local database first. If no local authority match was found, the Linker falls back to a remote Z39.50 search. >--- > C4/Biblio.pm | 11 +- > C4/Linker/Z3950Server.pm | 308 +++++++++++++++++++++ > cataloguing/automatic_linker.pl | 13 +- > .../en/modules/admin/preferences/authorities.pref | 5 + > .../prog/en/modules/cataloguing/addbiblio.tt | 16 ++ > 5 files changed, 347 insertions(+), 6 deletions(-) > create mode 100644 C4/Linker/Z3950Server.pm > >diff --git a/C4/Biblio.pm b/C4/Biblio.pm >index 6613ad4..dbd473b 100644 >--- a/C4/Biblio.pm >+++ b/C4/Biblio.pm >@@ -507,7 +507,7 @@ sub BiblioAutoLink { > > =head2 LinkBibHeadingsToAuthorities > >- my $num_headings_changed, %results = LinkBibHeadingsToAuthorities($linker, $marc, $frameworkcode, [$allowrelink, $verbose]); >+ my $num_headings_changed, %results = LinkBibHeadingsToAuthorities($linker, $marc, $frameworkcode, [$allowrelink, $verbose, $dont_auto_create]); > > Links bib headings to authority records by checking > each authority-controlled field in the C<MARC::Record> >@@ -530,6 +530,7 @@ sub LinkBibHeadingsToAuthorities { > my $frameworkcode = shift; > my $allowrelink = shift; > my $verbose = shift; >+ my $dont_auto_create = shift; > my %results; > if (!$bib) { > carp 'LinkBibHeadingsToAuthorities called on undefined bib record'; >@@ -555,6 +556,9 @@ sub LinkBibHeadingsToAuthorities { > } > > my ( $authid, $fuzzy, $status ) = $linker->get_link($heading); >+ if(defined $status and ($status eq 'NO_CONNECTION' or $status eq 'SERVER_NOT_FOUND')) { >+ return 0, { error => $status }; >+ } > if ($authid) { > $results{ $fuzzy ? 'fuzzy' : 'linked' } > ->{ $heading->display_form() }++; >@@ -575,9 +579,10 @@ sub LinkBibHeadingsToAuthorities { > $results{'fuzzy'}->{ $heading->display_form() }++; > push(@{$results{'details'}}, { tag => $field->tag(), authid => $current_link, status => 'UNCHANGED'}) if $verbose; > } >- elsif ( C4::Context->preference('AutoCreateAuthorities') ) { >+ elsif ( !$dont_auto_create && C4::Context->preference('AutoCreateAuthorities') ) { > if ( _check_valid_auth_link( $current_link, $field ) ) { > $results{'linked'}->{ $heading->display_form() }++; >+ push(@{$results{'details'}}, { tag => $field->tag(), authid => $authid, status => 'UNCHANGED'}) if $verbose; > } > else { > $authid = AddAuthorityFromHeading($heading, $field, $bib); >@@ -649,6 +654,7 @@ sub AddAuthorityFromHeading { > my $heading = shift; > my $field = shift; > my $bib = shift; >+ my $current_link = $field->subfield('9'); > > my $authtypedata = > C4::AuthoritiesMarc::GetAuthType( $heading->auth_type() ); >@@ -657,6 +663,7 @@ sub AddAuthorityFromHeading { > $marcrecordauth->leader(' nz a22 o 4500'); > SetMarcUnicodeFlag( $marcrecordauth, 'MARC21' ); > } >+ $field->delete_subfield( code => '9' ) if defined $current_link; > my $authfield = > MARC::Field->new( $authtypedata->{auth_tag_to_report}, > '', '', "a" => "" . $field->subfield('a') ); >diff --git a/C4/Linker/Z3950Server.pm b/C4/Linker/Z3950Server.pm >new file mode 100644 >index 0000000..fea3adf >--- /dev/null >+++ b/C4/Linker/Z3950Server.pm >@@ -0,0 +1,308 @@ >+package C4::Linker::Z3950Server; >+ >+# Copyright 2012 Frédérick Capovilla - Libéo >+# >+# Koha is free software; you can redistribute it and/or modify it >+# under the terms of the GNU General Public License as published by >+# the Free Software Foundation; either version 3 of the License, or >+# (at your option) any later version. >+# >+# Koha is distributed in the hope that it will be useful, but >+# WITHOUT ANY WARRANTY; without even the implied warranty of >+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the >+# GNU General Public License for more details. >+# >+# You should have received a copy of the GNU General Public License >+# along with Koha; if not, see <http://www.gnu.org/licenses>. >+ >+use strict; >+use warnings; >+use Carp; >+use MARC::Field; >+use MARC::Record; >+use C4::Context; >+use C4::Heading; >+use ZOOM; # For Z39.50 Searches >+use C4::AuthoritiesMarc; # For Authority addition >+use C4::Log; # logaction >+ >+use base qw(C4::Linker); >+ >+ >+sub get_link { >+ my $self = shift; >+ my $heading = shift; >+ my $behavior = shift || 'default'; >+ my $search_form = $heading->search_form(); >+ my $authid; >+ my $fuzzy = 0; >+ my $status = ''; >+ >+ if ( $self->{'cache'}->{$search_form}->{'cached'} ) { >+ $authid = $self->{'cache'}->{$search_form}->{'authid'}; >+ $fuzzy = $self->{'cache'}->{$search_form}->{'fuzzy'}; >+ } >+ else { >+ # Look for matching authorities on the Z39.50 server >+ unless($self->{'local_first'}) { >+ ($authid, undef, $status) = $self->getZ3950Authority($heading); >+ >+ if($status eq 'Z3950_SEARCH_ERROR' or >+ $status eq 'Z3950_QUERY_ERROR' or >+ $status eq 'NO_CONNECTION' or >+ $status eq 'SERVER_NOT_FOUND') { >+ return $authid, $fuzzy, $status; >+ } >+ } >+ >+ if(!$authid) { >+ # look for matching authorities >+ my $authorities = $heading->authorities(1); # $skipmetadata = true >+ >+ if ( $behavior eq 'default' && $#{$authorities} == 0 ) { >+ $authid = $authorities->[0]->{'authid'}; >+ } >+ elsif ( $behavior eq 'first' && $#{$authorities} >= 0 ) { >+ $authid = $authorities->[0]->{'authid'}; >+ $fuzzy = $#{$authorities} > 0; >+ } >+ elsif ( $behavior eq 'last' && $#{$authorities} >= 0 ) { >+ $authid = $authorities->[ $#{$authorities} ]->{'authid'}; >+ $fuzzy = $#{$authorities} > 0; >+ } >+ >+ if ( !defined $authid && $self->{'broader_headings'} ) { >+ my $field = $heading->field(); >+ my @subfields = $field->subfields(); >+ if ( scalar @subfields > 1 ) { >+ pop @subfields; >+ $field->replace_with( >+ MARC::Field->new( >+ $field->tag, >+ $field->indicator(1), >+ $field->indicator(2), >+ map { $_[0] => $_[1] } @subfields >+ ) >+ ); >+ ( $authid, $fuzzy ) = >+ $self->get_link( C4::Heading->new_from_bib_field($field), >+ $behavior ); >+ } >+ } >+ } >+ >+ if($self->{'local_first'} && !$authid) { >+ ($authid, undef, $status) = $self->getZ3950Authority($heading); >+ } >+ >+ $self->{'cache'}->{$search_form}->{'cached'} = 1; >+ $self->{'cache'}->{$search_form}->{'authid'} = $authid; >+ $self->{'cache'}->{$search_form}->{'fuzzy'} = $fuzzy; >+ } >+ return $self->SUPER::_handle_auth_limit($authid), $fuzzy, $status; >+} >+ >+sub flip_heading { >+ my $self = shift; >+ my $heading = shift; >+ >+ # TODO: implement >+} >+ >+ >+=head1 getZ3950Authority >+ >+ ($authid, $record, $status) = $self->getZ3950Authority($heading); >+ >+ Do a Z39.50 search for the heading using the $conn ZOOM::Connection object and the $heading Heading. >+ The column kx_rcimport_control_number is used to check for duplicates. >+ FIXME: Use thesaurus in search? As of Koha 3.8, the community stopped using them. >+ >+ RETURNS : >+ $authid = the ID of the local copy of the authority record that was found. undef if nothing was found. >+ $record = the MARC record of the found authority. >+ $status = A string with additional informations on the search status (Z3950_CREATED, Z3950_UPDATED, Z3950_QUERY_ERROR, Z3950_SEARCH_ERROR) >+ >+=cut >+ >+sub getZ3950Authority { >+ my $self = shift; >+ my $heading = shift; >+ >+ # Try to find a match on the Z39.50 server if LinkerZ3950Server is set >+ if(C4::Context->preference('LinkerZ3950Server')) { >+ unless($self->{'conn'}) { >+ my $sth = C4::Context->dbh->prepare("select * from z3950servers where name=?"); >+ $sth->execute(C4::Context->preference('LinkerZ3950Server')); >+ my $server = $sth->fetchrow_hashref or undef; >+ $sth->finish; >+ >+ if($server) { >+ my $options = new ZOOM::Options(); >+ $options->option('cclfile' => C4::Context->ModZebrations('authorityserver')->{"ccl2rpn"}); >+ $options->option('elementSetName', 'F'); >+ $options->option('async', 0); >+ $options->option('databaseName', $server->{db}); >+ $options->option('user', $server->{userid} ) if $server->{userid}; >+ $options->option('password', $server->{password}) if $server->{password}; >+ $options->option('preferredRecordSyntax', $server->{syntax}); >+ $self->{'conn'} = create ZOOM::Connection($options); >+ eval{ $self->{'conn'}->connect( $server->{host}, $server->{port} ) }; >+ if($@) { >+ return (undef, undef, 'NO_CONNECTION'); >+ } >+ } >+ else { >+ return (undef, undef, 'SERVER_NOT_FOUND'); >+ } >+ } >+ } >+ else { >+ return; >+ } >+ >+ my $query = qq(Match-heading,do-not-truncate,ext="$heading->{'search_form'}"); >+ >+ my $zquery = eval{ new ZOOM::Query::CCL2RPN($query, $self->{'conn'}) }; >+ if($@) { >+ warn $query . "\n" . $@; >+ return (undef, undef, 'Z3950_QUERY_ERROR'); >+ } >+ >+ # Try to send the search query to the server. >+ my $rs = eval{ $self->{'conn'}->search($zquery) }; >+ if($@){ >+ warn $query . "\n" . $@; >+ return (undef, undef, 'Z3950_SEARCH_ERROR'); >+ } >+ >+ # If authorities are found, select the first valid one for addition in the local authority table. >+ my $record; >+ if($rs->size != 0) { >+ $record = MARC::Record::new_from_usmarc($rs->record(0)->raw()); >+ } >+ $rs->destroy(); >+ $zquery->destroy(); >+ >+ # If a valid authority was found, add it in the local authority database. >+ if($record) { >+ my $dbh=C4::Context->dbh; >+ >+ my $authtypecode = C4::AuthoritiesMarc::GuessAuthTypeCode($record); >+ my $authId; >+ >+ # Use the control number to prevent the creation of duplicate authorities. >+ my $controlNumber = $record->field('970')->subfield('0'); >+ my $sthExist=$dbh->prepare("SELECT authid FROM auth_header WHERE kx_rcimport_control_number=?"); >+ $sthExist->execute($controlNumber) or die $sthExist->errstr; >+ ($authId) = $sthExist->fetchrow; >+ $sthExist->finish; >+ >+ #------------------------------------------------------------------------------------------ >+ # Corrections and verifications before insertion >+ my $format; >+ my $leader=' nz a22 o 4500';# Leader for incomplete MARC21 record >+ >+ if (uc(C4::Context->preference('marcflavour')) eq 'UNIMARC') { >+ $format= 'UNIMARCAUTH'; >+ } >+ else { >+ $format= 'MARC21'; >+ } >+ >+ if ($format eq "MARC21") { >+ if (!$record->leader) { >+ $record->leader($leader); >+ } >+ if (!$record->field('003')) { >+ $record->insert_fields_ordered( >+ MARC::Field->new('003',C4::Context->preference('MARCOrgCode')) >+ ); >+ } >+ my $time=POSIX::strftime("%Y%m%d%H%M%S",localtime); >+ if (!$record->field('005')) { >+ $record->insert_fields_ordered( >+ MARC::Field->new('005',$time.".0") >+ ); >+ } >+ my $date=POSIX::strftime("%y%m%d",localtime); >+ if (!$record->field('008')) { >+ # Get a valid default value for field 008 >+ my $default_008 = C4::Context->preference('MARCAuthorityControlField008'); >+ if(!$default_008 or length($default_008)<34) { >+ $default_008 = '|| aca||aabn | a|a d'; >+ } >+ else { >+ $default_008 = substr($default_008,0,34); >+ } >+ >+ $record->insert_fields_ordered( MARC::Field->new('008',$date.$default_008) ); >+ } >+ if (!$record->field('040')) { >+ $record->insert_fields_ordered( >+ MARC::Field->new('040','','', >+ 'a' => C4::Context->preference('MARCOrgCode'), >+ 'c' => C4::Context->preference('MARCOrgCode') >+ ) >+ ); >+ } >+ } >+ if ($format eq "UNIMARCAUTH") { >+ $record->leader(" nx j22 ") unless ($record->leader()); >+ my $date=POSIX::strftime("%Y%m%d",localtime); >+ if (my $string=$record->subfield('100',"a")){ >+ $string=~s/fre50/frey50/; >+ $record->field('100')->update('a'=>$string); >+ } >+ elsif ($record->field('100')){ >+ $record->field('100')->update('a'=>$date."afrey50 ba0"); >+ } else { >+ $record->append_fields( >+ MARC::Field->new('100',' ',' ' >+ ,'a'=>$date."afrey50 ba0") >+ ); >+ } >+ } >+ my ($auth_type_tag, $auth_type_subfield) = C4::AuthoritiesMarc::get_auth_type_location($authtypecode); >+ if (!$authId and $format eq "MARC21") { >+ # only need to do this fix when modifying an existing authority >+ C4::AuthoritiesMarc::MARC21::fix_marc21_auth_type_location($record, $auth_type_tag, $auth_type_subfield); >+ } >+ if (my $field=$record->field($auth_type_tag)){ >+ $field->update($auth_type_subfield=>$authtypecode); >+ } >+ else { >+ $record->add_fields($auth_type_tag,'','', $auth_type_subfield=>$authtypecode); >+ } >+ #------------------------------------------------------------------------------------------ >+ if ($authId) { >+ my $oldRecord=C4::AuthoritiesMarc::GetAuthority($authId); >+ >+ my $sth=$dbh->prepare("UPDATE auth_header SET authtypecode=?,marc=?,marcxml=? WHERE kx_rcimport_control_number=?"); >+ eval { $sth->execute($authtypecode,$record->as_usmarc,$record->as_xml_record($format),$controlNumber) or die $sth->errstr; }; >+ $sth->finish; >+ warn "Problem with authority $controlNumber : Cannot update" if $@; >+ $dbh->commit unless $dbh->{AutoCommit}; >+ return if $@; >+ >+ C4::Biblio::ModZebra($authId,'linkerUpdate',"authorityserver",$oldRecord,$record); >+ return ($authId, $record, 'Z3950_UPDATED'); >+ } >+ else { >+ my $sth=$dbh->prepare("INSERT INTO auth_header (datecreated,authtypecode,marc,marcxml,kx_rcimport_control_number) VALUES (NOW(),?,?,?,?)"); >+ eval { $sth->execute($authtypecode,$record->as_usmarc,$record->as_xml_record($format),$controlNumber) or die $sth->errstr; }; >+ $sth->finish; >+ warn "Problem with authority $controlNumber : Cannot insert" if $@; >+ my $id = $dbh->{'mysql_insertid'}; >+ $dbh->commit unless $dbh->{AutoCommit}; >+ return if $@; >+ >+ logaction( "AUTHORITIES", "ADD", $id, "authority" ) if C4::Context->preference("AuthoritiesLog"); >+ C4::Biblio::ModZebra($id,'linkerUpdate',"authorityserver",undef,$record); >+ return ($id, $record, 'Z3950_CREATED'); >+ } >+ } >+ return; >+} >+ >+1; >diff --git a/cataloguing/automatic_linker.pl b/cataloguing/automatic_linker.pl >index d99e9f3..7561691 100755 >--- a/cataloguing/automatic_linker.pl >+++ b/cataloguing/automatic_linker.pl >@@ -43,7 +43,7 @@ if ($auth_status ne "ok") { > # > # tag = the tag number of the field > # authid = the value of the $9 subfield for this tag >-# status = The status of the link (LOCAL_FOUND, NONE_FOUND, MULTIPLE_MATCH, UNCHANGED, CREATED) >+# status = The status of the link (LOCAL_FOUND, NONE_FOUND, MULTIPLE_MATCH, UNCHANGED, CREATED, Z3950_CREATED, Z3950_UPDATED) > > my $json; > >@@ -62,9 +62,14 @@ if ($@) { > } > my $linker = $linker_module->new( { 'options' => C4::Context->preference("LinkerOptions") } ); > >-my ( $headings_changed, $results ) = LinkBibHeadingsToAuthorities( $linker, $record, $params['frameworkcode'], C4::Context->preference("CatalogModuleRelink") || '', 1 ); >+my ( $headings_changed, $results ) = LinkBibHeadingsToAuthorities( $linker, $record, $params['frameworkcode'], C4::Context->preference("CatalogModuleRelink") || '', 1, 1 ); > >-$json->{status} = 'OK'; >-$json->{links} = $results->{details} || ''; >+if(defined $results->{error}) { >+ $json->{status} = $results->{error}; >+} >+else { >+ $json->{status} = 'OK'; >+ $json->{links} = $results->{details} || ''; >+} > > print to_json($json); >diff --git a/koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/authorities.pref b/koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/authorities.pref >index 2762411..443877b 100644 >--- a/koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/authorities.pref >+++ b/koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/authorities.pref >@@ -58,6 +58,7 @@ Authorities: > Default: Default > FirstMatch: "First Match" > LastMatch: "Last Match" >+ Z3950Server: "Z39.50 Server" > - linker module for matching headings to authority records. > - > - Set the following options for the authority linker >@@ -85,3 +86,7 @@ Authorities: > yes: Do > no: "Do not" > - automatically relink headings that have previously been linked when saving records in the cataloging module. >+ - >+ - "Import authorities from this Z39.50 server when searching for authority links with the 'Z39.50 Server' linker module :" >+ - pref: LinkerZ3950Server >+ class: multi >diff --git a/koha-tmpl/intranet-tmpl/prog/en/modules/cataloguing/addbiblio.tt b/koha-tmpl/intranet-tmpl/prog/en/modules/cataloguing/addbiblio.tt >index 4c36ce4..0c5f33d 100644 >--- a/koha-tmpl/intranet-tmpl/prog/en/modules/cataloguing/addbiblio.tt >+++ b/koha-tmpl/intranet-tmpl/prog/en/modules/cataloguing/addbiblio.tt >@@ -231,6 +231,16 @@ function updateHeadingLinks(links) { > var message = ''; > var field_color = '#FFAAAA'; > switch(heading.status) { >+ case 'Z3950_CREATED': >+ image = 'approve.gif'; >+ message = _("A matching authority record was found on the Z39.50 server and was imported locally."); >+ field_color = '#99FF99'; >+ break; >+ case 'Z3950_UPDATED': >+ image = 'approve.gif'; >+ message = _("A matching authority record was found on the Z39.50 server and a local authority was updated."); >+ field_color = '#99FF99'; >+ break; > case 'LOCAL_FOUND': > image = 'approve.gif'; > message = _("A matching authority was found in the local database."); >@@ -312,6 +322,12 @@ function AutomaticLinker() { > case 'UNAUTHORIZED': > alert(_("Error : You do not have the permissions necessary to use this functionality.")); > break; >+ case 'SERVER_NOT_FOUND': >+ alert(_("Error : The Z39.50 server configured in the 'LinkerZ3950Server' preference was not found in the Z39.50 servers list.")); >+ break; >+ case 'NO_CONNECTION': >+ alert(_("Error : Could not connect to the Z39.50 server.")); >+ break; > case 'OK': > updateHeadingLinks(json.links); > break; >-- >2.1.0
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Diff
|
Splinter Review
Attachments on
bug 11300
:
23147
|
37918
|
37974
|
37975
|
37989
|
39649
|
45324
|
56542
|
56543
|
60999
|
67165
|
69096
|
69204
|
79400
|
94819
|
94875
|
111801
|
111896
|
115217
|
133496
|
136404
|
138029
|
141168
|
143292
|
148824
|
151079
|
151476
|
151477
|
155602
|
155603
|
159741
|
159742
|
163946
|
163947
|
171304
|
171689