Bugzilla – Attachment 142727 Details for
Bug 31969
Options for cleanup_database.pl to remove finished jobs from the background job queue
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Help
|
New Account
|
Log In
[x]
|
Forgot Password
Login:
[x]
[patch]
Bug 31969: Add background queue options to cleanup_database
Bug-31969-Add-background-queue-options-to-cleanupd.patch (text/plain), 7.54 KB, created by
Tomás Cohen Arazi (tcohen)
on 2022-10-27 18:00:30 UTC
(
hide
)
Description:
Bug 31969: Add background queue options to cleanup_database
Filename:
MIME Type:
Creator:
Tomás Cohen Arazi (tcohen)
Created:
2022-10-27 18:00:30 UTC
Size:
7.54 KB
patch
obsolete
>From d27854761d96365d193feb113a05a8d339fc9670 Mon Sep 17 00:00:00 2001 >From: =?UTF-8?q?Bj=C3=B6rn=20Nyl=C3=A9n?= <bjorn.nylen@ub.lu.se> >Date: Mon, 24 Oct 2022 15:12:03 +0200 >Subject: [PATCH] Bug 31969: Add background queue options to cleanup_database > >This patch adds background queue options to cleanup_database.pl to allow >for purging completed background jobs. > >--bg-jobs DAYS Purge all finished background jobs this many days old. Defaults to 1 if no DAYS provided. >--bg-type TYPE What type of background job to purge. Defaults to "update_elastic_index" if omitted > Specifying "all" will purge all types. Repeatable. > >To test: > 1 - Enable elastic search in Koha > 2 - perl misc/maintenance/touch_all_items.pl > 3 - Generate an number of diffrent types of bg-jobs (eg batch_hold_cancel, > batch_biblio_record_deletion, batch_item_record_deletion) > 4 - Check db and note there are a bunch of diffrent jobs > 5 - Update to make them old > UPDATE background_jobs SET ended_on = '2022-10-01 00:00:00', status='finished' > 6 - perl misc/cronjobs/cleanup_database.pl > 7 - Note bg-jobs entry shows in help > 8 - perl misc/cronjobs/cleanup_database.pl --bg-jobs 1 -v > 9 - Note that elasticqueue would have been cleared >10 - perl misc/cronjobs/cleanup_database.pl --bg-jobs 1 -v --confirm >11 - Note that number of entries deleted is reported >12 - Attempt to clear other job types, including "all" eg > perl misc/cronjobs/cleanup_database.pl --bg-jobs 1 --bg-type batch_item_record_deletion -v --confirm > perl misc/cronjobs/cleanup_database.pl --bg-jobs 1 --bg-type all -v --confirm >13 - Confirm in staff interface that jobs are gone: > http://localhost:8081/cgi-bin/koha/admin/background_jobs.pl > (Uncheck 'Current jobs only') > >Signed-off-by: Nick Clemens <nick@bywatersolutions.com> >Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> >--- > misc/cronjobs/cleanup_database.pl | 50 ++++++++++++++++++++++++++++++- > 1 file changed, 49 insertions(+), 1 deletion(-) > >diff --git a/misc/cronjobs/cleanup_database.pl b/misc/cronjobs/cleanup_database.pl >index dc62ef8aea..612ae3639b 100755 >--- a/misc/cronjobs/cleanup_database.pl >+++ b/misc/cronjobs/cleanup_database.pl >@@ -27,6 +27,8 @@ use constant DEFAULT_MESSAGES_PURGEDAYS => 365; > use constant DEFAULT_SEARCHHISTORY_PURGEDAYS => 30; > use constant DEFAULT_SHARE_INVITATION_EXPIRY_DAYS => 14; > use constant DEFAULT_DEBARMENTS_PURGEDAYS => 30; >+use constant DEFAULT_BGJOBS_PURGEDAYS => 1; >+use constant DEFAULT_BGJOBS_PURGETYPES => qw{ update_elastic_index }; > > use Koha::Script -cron; > use C4::Context; >@@ -36,6 +38,7 @@ use Getopt::Long qw( GetOptions ); > use C4::Log qw( cronlogaction ); > use C4::Accounts qw( purge_zero_balance_fees ); > use Koha::UploadedFiles; >+use Koha::BackgroundJobs; > use Koha::Old::Biblios; > use Koha::Old::Items; > use Koha::Old::Biblioitems; >@@ -49,7 +52,7 @@ use Koha::Patron::Debarments qw( DelDebarment ); > > sub usage { > print STDERR <<USAGE; >-Usage: $0 [-h|--help] [--confirm] [--sessions] [--sessdays DAYS] [-v|--verbose] [--zebraqueue DAYS] [-m|--mail] [--merged] [--import DAYS] [--logs DAYS] [--searchhistory DAYS] [--restrictions DAYS] [--all-restrictions] [--fees DAYS] [--temp-uploads] [--temp-uploads-days DAYS] [--uploads-missing 0|1 ] [--statistics DAYS] [--deleted-catalog DAYS] [--deleted-patrons DAYS] [--old-issues DAYS] [--old-reserves DAYS] [--transfers DAYS] [--labels DAYS] [--cards DAYS] >+Usage: $0 [-h|--help] [--confirm] [--sessions] [--sessdays DAYS] [-v|--verbose] [--zebraqueue DAYS] [-m|--mail] [--merged] [--import DAYS] [--logs DAYS] [--searchhistory DAYS] [--restrictions DAYS] [--all-restrictions] [--fees DAYS] [--temp-uploads] [--temp-uploads-days DAYS] [--uploads-missing 0|1 ] [--statistics DAYS] [--deleted-catalog DAYS] [--deleted-patrons DAYS] [--old-issues DAYS] [--old-reserves DAYS] [--transfers DAYS] [--labels DAYS] [--cards DAYS] [--bg-jobs DAYS [--bg-type TYPE] ] > > -h --help prints this help message, and exits, ignoring all > other options >@@ -107,6 +110,9 @@ Usage: $0 [-h|--help] [--confirm] [--sessions] [--sessdays DAYS] [-v|--verbose] > --cards DAY Purge card creator batches last added to more than DAYS days ago. > --return-claims Purge all resolved return claims older than the number of days specified in > the system preference CleanUpDatabaseReturnClaims. >+ --bg-days DAYS Purge all finished background jobs this many days old. Defaults to 1 if no DAYS provided. >+ --bg-type TYPES What type of background job to purge. Defaults to "update_elastic_index" if omitted >+ Specifying "all" will purge all types. Repeatable. > USAGE > exit $_[0]; > } >@@ -148,6 +154,8 @@ my $labels; > my $cards; > my @log_modules; > my @preserve_logs; >+my $background_days; >+my @background_types; > > my $command_line_options = join(" ",@ARGV); > >@@ -190,6 +198,8 @@ GetOptions( > 'labels' => \$labels, > 'cards' => \$cards, > 'return-claims' => \$return_claims, >+ 'bg-type:s' => \@background_types, >+ 'bg-jobs:i' => \$background_days, > ) || usage(1); > > # Use default values >@@ -202,6 +212,8 @@ $pSearchhistory = DEFAULT_SEARCHHISTORY_PURGEDAYS if defined($pSearchhis > $pListShareInvites = DEFAULT_SHARE_INVITATION_EXPIRY_DAYS if defined($pListShareInvites) && $pListShareInvites == 0; > $pDebarments = DEFAULT_DEBARMENTS_PURGEDAYS if defined($pDebarments) && $pDebarments == 0; > $pMessages = DEFAULT_MESSAGES_PURGEDAYS if defined($pMessages) && $pMessages == 0; >+$background_days = DEFAULT_BGJOBS_PURGEDAYS if defined($background_days) && $background_days == 0; >+@background_types = (DEFAULT_BGJOBS_PURGETYPES) if $background_days && @background_types == 0; > > if ($help) { > usage(0); >@@ -239,6 +251,7 @@ unless ( $sessions > || $labels > || $cards > || $return_claims >+ || $background_days > ) { > print "You did not specify any cleanup work for the script to do.\n\n"; > usage(1); >@@ -675,6 +688,16 @@ if ($cards) { > } > } > >+if ($background_days) { >+ print "Purging background jobs more than $background_days days ago.\n" if $verbose; >+ my $count = PurgeBackgroundJobs($background_days, \@background_types, $confirm); >+ if ($verbose) { >+ say $confirm >+ ? sprintf "Done with purging %d background jobs of type(s): %s added more than %d days ago.\n", $count, join(',', @background_types), $background_days >+ : sprintf "%d background jobs of type(s): %s added more than %d days ago would have been purged.", $count, join(',', @background_types), $background_days; >+ } >+} >+ > cronlogaction({ action => 'End', info => "COMPLETED" }); > > exit(0); >@@ -812,3 +835,28 @@ sub DeleteSpecialHolidays { > my $count = $sth->execute( $days ) + 0; > print "Removed $count unique holidays\n" if $verbose; > } >+ >+sub PurgeBackgroundJobs { >+ my ( $days, $types, $confirm ) = @_; >+ >+ my $rs; >+ if ( $types->[0] eq 'all' ){ >+ $rs = Koha::BackgroundJobs->search( >+ { >+ ended_on => { '<' => \[ 'date_sub(curdate(), INTERVAL ? DAY)', $days ] }, >+ status => 'finished', >+ }); >+ >+ } else { >+ $rs = Koha::BackgroundJobs->search( >+ { >+ ended_on => { '<' => \[ 'date_sub(curdate(), INTERVAL ? DAY)', $days ] }, >+ type => \@{$types}, >+ status => 'finished', >+ }); >+ } >+ my $count = $rs->count(); >+ $rs->delete if $confirm; >+ >+ return $count; >+} >-- >2.34.1
<b>You cannot view the attachment while viewing its details because your browser does not support IFRAMEs. <a href="attachment.cgi?id=142727">View the attachment on a separate page</a>.</b>
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Diff
|
Splinter Review
Attachments on
bug 31969
:
142603
|
142668
|
142699
|
142700
| 142727 |
142728
|
142729
|
144189
|
144190