Bugzilla – Attachment 44446 Details for
Bug 15104
Batch Record Modification Performance Improvement
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Help
|
New Account
|
Log In
[x]
|
Forgot Password
Login:
[x]
[patch]
Bug 15104: Batch Record Modification Performance Improvement
Bug-15104-Batch-Record-Modification-Performance-Im.patch (text/plain), 6.46 KB, created by
cnorthcott.work
on 2015-11-05 01:30:44 UTC
(
hide
)
Description:
Bug 15104: Batch Record Modification Performance Improvement
Filename:
MIME Type:
Creator:
cnorthcott.work
Created:
2015-11-05 01:30:44 UTC
Size:
6.46 KB
patch
obsolete
>From 540853f34ac42b749fc5628f69ec9199768f3b80 Mon Sep 17 00:00:00 2001 >From: Catherine <cnorthcott.work@gmail.com> >Date: Tue, 3 Nov 2015 02:31:57 +0000 >Subject: [PATCH] Bug 15104: Batch Record Modification Performance Improvement > >Performance improvements to speed of the Modify Records by decreasing unnecessary >progress reporting of success messages and data fetching while logging. > >Test Plan: >1) Prepare NTYprof (http://wiki.koha-community.org/wiki/Profiling_with_Devel::NYTProf) > or procure a stopwatch (stopwatch will be simpler but less accurate). >2) Ensure you have a decent number of records in your Koha system (>500) if you are > lacking records to modify in your database you can import some from files > found here: http://wiki.koha-community.org/wiki/Free_sample_MARC_data >3) Prepare a file with biblionumbers to modify. You can create a file containing > all record called ItemBarcodes.txt in the current folder using the > following command from the machine running Koha: > echo 'select biblionumber from biblio' | sudo koha-mysql kohadev >> BiblioNums.txt >4) Navigate to Home > Tools > MARC modification templates in the Koha Intranet. >5) Create a new template called ModRec1. >6) Add a new action using: > Select 'Add/Update' from the drop down box and in the field(s) boxes enter '090' > (in the first box) and 'a' (in the second box) and 'Mod1' in the value field. > Then click 'Add Action' >7) Create a new template called ModRec2. >8) Add a new action using: > Select 'Add/Update' from the drop down box and in the field(s) boxes enter '090' > (in the first box) and 'a' (in the second box) and 'Mod2' in the value field. > Then click 'Add Action' >9) Navigate to Home > Tools > Batch Modify Records in the Koha Intranet >10) Click the 'Browse...' button and select BiblioNums.txt or your equilient. >11) Select ModRec1 under Use MARC Modification Template. >12) Click the 'Continue' button and scroll to the bottom of the page. >13) If you are using a stopwatch, prepare your stopwatch so that you will > start counting seconds from the point you click the button in the next > step. >14) Click the 'Modify Selected Records' button (simultaneously start your > stopwatch if using one) >15) When the page appears showing completion of the change, stop your stopwatch > and check the time or navigate to the folder you have set NYTProf to output. >16) Record the runtime. This is the pre-optimisation time. >17) If this is the pre-patch test then you should see lots of messages like: > "The biblio #### has successfully been modified." > If this is the post-patch test then you should see only: > "All records have successfully been modified!" > and a link to do another record modification. >18) Apply this patch. >19) Repeat steps 9-12 of this testplan with the patch applied, replacing ModRec1 > in step 11 with ModRec2. This will yield the post-optimisation time. If you > are using NYTProf you will need to apply the -d:NYTProf shebang to the file > again. >20) Compare the pre-optimisation time and post optimisation time. The second > post-optimisation time should be faster. >--- > C4/Biblio.pm | 3 +-- > tools/batch_record_modification.pl | 38 ++++++++++++++++++------------------ > 2 files changed, 20 insertions(+), 21 deletions(-) > >diff --git a/C4/Biblio.pm b/C4/Biblio.pm >index 2e8827a..42a85ae 100644 >--- a/C4/Biblio.pm >+++ b/C4/Biblio.pm >@@ -307,8 +307,7 @@ sub ModBiblio { > } > > if ( C4::Context->preference("CataloguingLog") ) { >- my $newrecord = GetMarcBiblio($biblionumber); >- logaction( "CATALOGUING", "MODIFY", $biblionumber, "biblio BEFORE=>" . $newrecord->as_formatted ); >+ logaction( "CATALOGUING", "MODIFY", $biblionumber, "biblio BEFORE=>" . $record->as_formatted ); > } > > # Cleaning up invalid fields must be done early or SetUTF8Flag is liable to >diff --git a/tools/batch_record_modification.pl b/tools/batch_record_modification.pl >index 60775a9..f752dee 100755 >--- a/tools/batch_record_modification.pl >+++ b/tools/batch_record_modification.pl >@@ -153,10 +153,11 @@ if ( $op eq 'form' ) { > # We want to modify selected records! > my @record_ids = $input->param('record_id'); > >+ my $job_size; > my ( $job ); > if ( $runinbackground ) { >- my $job_size = scalar( @record_ids ); >- $job = C4::BackgroundJob->new( $sessionID, "FIXME", '/cgi-bin/koha/tools/batch_record_modification.pl', $job_size ); >+ $job_size = scalar( @record_ids ); >+ $job = C4::BackgroundJob->new( $sessionID, "FIXME", $ENV{SCRIPT_NAME}, $job_size ); > my $job_id = $job->id; > if (my $pid = fork) { > $dbh->{InactiveDestroy} = 1; >@@ -178,9 +179,12 @@ if ( $op eq 'form' ) { > total_records => 0, > total_success => 0, > }; >+ my $has_errors; > my $progress = 0; > $dbh->{RaiseError} = 1; > RECORD_IDS: for my $record_id ( sort { $a <=> $b } @record_ids ) { >+ $progress++; >+ $has_errors = 0; > $report->{total_records}++; > next unless $record_id; > >@@ -201,12 +205,8 @@ if ( $op eq 'form' ) { > biblionumber => $biblionumber, > error => ($@ ? $@ : $error), > }; >+ $has_errors++; > } else { >- push @messages, { >- type => 'success', >- code => 'biblio_modified', >- biblionumber => $biblionumber, >- }; > $report->{total_success}++; > } > } else { >@@ -225,22 +225,22 @@ if ( $op eq 'form' ) { > authid => $authid, > error => ($@ ? $@ : 0), > }; >+ $has_errors++; > } else { >- push @messages, { >- type => 'success', >- code => 'authority_modified', >- authid => $authid, >- }; > $report->{total_success}++; > } > } >- >- $job->set({ >- view => 'report', >- report => $report, >- messages => \@messages, >- }); >- $job->progress( ++$progress ) if $runinbackground; >+ >+ if ($has_errors){ >+ $job->set({ >+ view => 'report', >+ report => $report, >+ messages => \@messages, >+ }); >+ } >+ >+ $job->progress( ++$progress ) if $runinbackground && (!($progress%10) || ($progress==$job_size)); >+ > } > > if ($runinbackground) { >-- >1.7.10.4
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Diff
|
Splinter Review
Attachments on
bug 15104
: 44446