Bugzilla – Attachment 82558 Details for
Bug 21865
Add Elasticsearch support to, and improve verbose output of, `remove_unused_authorities.pl`
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Help
|
New Account
|
Log In
[x]
|
Forgot Password
Login:
[x]
[patch]
Bug 21865: improve remove_unused_authorities.pl script
Bug-21865-improve-removeunusedauthoritiespl-script.patch (text/plain), 5.18 KB, created by
Fridolin Somers
on 2018-11-21 08:17:56 UTC
(
hide
)
Description:
Bug 21865: improve remove_unused_authorities.pl script
Filename:
MIME Type:
Creator:
Fridolin Somers
Created:
2018-11-21 08:17:56 UTC
Size:
5.18 KB
patch
obsolete
>From 55079c40585a043e6322e67d296644d2d87a2068 Mon Sep 17 00:00:00 2001 >From: Fridolin Somers <fridolin.somers@biblibre.com> >Date: Mon, 19 Nov 2018 14:46:05 +0100 >Subject: [PATCH] Bug 21865: improve remove_unused_authorities.pl script > >remove_unused_authorities.pl script can be improved. > >This patch changes changes verbosity so than test mode can be used >to know the autorities that are used and those that can be deleted. >It also writes a line in output if limited authority type(s). > >This patch also removes the unused vars $thresholdmin and $thresholdmax. > >It also changes the query to use SQL with parameters for authority types. > >Test plan : >1) On a catalog create a new authority >2) Be sure catalog is well indexed >3) Run misc/migration_tools/remove_unused_authorities.pl -t >4) You will see the line : > *** Testing only, authorities will not be deleted. *** >5) You will see lines of : > authid=x type=y : used X time(s) >6) You will see the line for the authority created in 1) : > authid=x type=y : can be deleted >7) You will see at the end : > x authorities parsed > y can be deleted because unused > z unchanged because used >8) Run misc/migration_tools/remove_unused_authorities.pl >9) You don't see the line : > *** Testing only, authorities will not be deleted. *** >10) You will see lines of : > authid=x type=y : used X time(s) >11) You will see the line for the authority created in 1) : > authid=x type=y : deleted >12) You will see at the end : > x authorities parsed > y deleted because unused > z unchanged because used >13) Run misc/migration_tools/remove_unused_authorities.pl --auth NP --auth CO >14) You see the line : > Restricted to authority type(s) : NP,CO. >--- > .../remove_unused_authorities.pl | 53 +++++++++++-------- > 1 file changed, 32 insertions(+), 21 deletions(-) > >diff --git a/misc/migration_tools/remove_unused_authorities.pl b/misc/migration_tools/remove_unused_authorities.pl >index f4e0ecbf2d..eabdcba5fd 100755 >--- a/misc/migration_tools/remove_unused_authorities.pl >+++ b/misc/migration_tools/remove_unused_authorities.pl >@@ -20,8 +20,7 @@ > # You should have received a copy of the GNU General Public License > # along with Koha; if not, see <http://www.gnu.org/licenses>. > >-use strict; >-use warnings; >+use Modern::Perl; > > use C4::Context; > use C4::AuthoritiesMarc; >@@ -42,55 +41,67 @@ if ($want_help) { > print_usage(); > exit 0; > } >- > if ($test) { >- print "testing only, authorities will not be deleted.\n"; >+ print "*** Testing only, authorities will not be deleted. ***\n"; >+} >+if (@authtypes) { >+ print "Restricted to authority type(s) : ".join(',', @authtypes).".\n"; > } > > my $errZebraConnection = C4::Context->Zconn("biblioserver",0)->errcode(); > if ( $errZebraConnection == 10000 ) { >- die "Zebra server seems not to be available. This script needs Zebra runs." >+ die "Zebra server seems not to be available. This script needs Zebra runs."; > } elsif ( $errZebraConnection ) { > die "Error from Zebra: $errZebraConnection"; > } > > my $dbh=C4::Context->dbh; >-my $thresholdmin=0; >-my $thresholdmax=0; > my @results; > # prepare the request to retrieve all authorities of the requested types >-my $rqsql = "SELECT * from auth_header where 1"; >-$rqsql .= " AND authtypecode IN (".join(",",map{$dbh->quote($_)}@authtypes).")" if @authtypes; >+my $rqsql = q{ SELECT authid,authtypecode FROM auth_header }; >+$rqsql .= q{ WHERE authtypecode IN (}.join(',',map{ '?' }@authtypes).')' if @authtypes; > my $rqselect = $dbh->prepare($rqsql); > $|=1; > >-$rqselect->execute; >+$rqselect->execute(@authtypes); > my $counter=0; > my $totdeleted=0; > my $totundeleted=0; > my $searcher = Koha::SearchEngine::Search->new({index => 'biblios'}); > while (my $data=$rqselect->fetchrow_hashref){ >- my $query; >- $query= "an=".$data->{'authid'}; >+ $counter++; >+ print 'authid='.$data->{'authid'}; >+ print ' type='.$data->{'authtypecode'}; >+ my $bibliosearch = 'an:'.$data->{'authid'}; > # search for biblios mapped >- my ($err,$res,$used) = $searcher->simple_search_compat($query,0,10); >+ my ($err,$res,$used) = $searcher->simple_search_compat($bibliosearch,0,10); > if (defined $err) { >- warn "error: $err on search $query\n"; >+ print "\n"; >+ warn "Error: $err on search for biblios $bibliosearch\n"; > next; > } >- print "."; >- print "$counter\n" unless $counter++ % 100; >- # if found, delete, otherwise, just count >- if ($used>=$thresholdmin and $used<=$thresholdmax){ >- DelAuthority({ authid => $data->{'authid'} }) unless $test; >+ unless ($used > 0){ >+ unless ($test) { >+ DelAuthority({ authid => $data->{'authid'} }); >+ print " : deleted"; >+ } else { >+ print " : can be deleted"; >+ } > $totdeleted++; > } else { > $totundeleted++; >+ print " : used $used time(s)"; > } >+ print "\n"; > } > >-print "$counter authorities parsed, $totdeleted deleted and $totundeleted unchanged because used\n"; >- >+print "$counter authorities parsed\n"; >+unless ($test) { >+ print "$totdeleted deleted because unused\n"; >+} else { >+ print "$totdeleted can be deleted because unused\n"; >+} >+print "$totundeleted unchanged because used\n"; > > sub print_usage { > print <<_USAGE_; >-- >2.17.1
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Diff
|
Splinter Review
Attachments on
bug 21865
:
82558
|
82559
|
82560
|
100577
|
100918
|
100919
|
100920
|
100921
|
102836
|
102837
|
102838
|
102839