Bug 18631 - `cleanup_database.pl` should take an option for modules in action_logs
Summary: `cleanup_database.pl` should take an option for modules in action_logs
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Command-line Utilities (show other bugs)
Version: Main
Hardware: All All
: P5 - low enhancement
Assignee: Nick Clemens (kidclamp)
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-05-18 18:17 UTC by Nick Clemens (kidclamp)
Modified: 2022-08-11 21:11 UTC (History)
10 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact: Caroline Cyr La Rose
Documentation submission: https://gitlab.com/koha-community/koha-manual/-/merge_requests/583
Text to go in the release notes:
This patch adds two new optional parameters to the `cleanup_database.pl` script. `--log-modules` - A repeatable option to specify which action log module lines to truncate. `--preserve-log` - A repeatable option to specify which action log module lines to keep.
Version(s) released in:
21.11.00
Circulation function:

Attachments
Bug 18631: Add option to specify logs to preserve or delete in cleanup_database.pl (4.18 KB, patch)
2021-07-27 18:50 UTC, Nick Clemens (kidclamp) 		Details | Diff | Splinter Review
Bug 18631: Add option to specify logs to preserve or delete in cleanup_database.pl (4.33 KB, patch)
2021-08-16 19:00 UTC, Andrew Fuerste-Henry 		Details | Diff | Splinter Review
Bug 18631: Add option to specify logs to preserve or delete in cleanup_database.pl (3.39 KB, patch)
2021-09-30 10:45 UTC, Martin Renvoize (ashimema) 		Details | Diff | Splinter Review
Bug 18631: Use - instead of _ for options (1.77 KB, patch)
2021-10-07 09:12 UTC, Jonathan Druart 		Details | Diff | Splinter Review
Show Obsolete (2) View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Clemens (kidclamp) 2017-05-18 18:17:48 UTC 
Cataloging log is a big one that often needs to be cleaned, other logs can be useful to keep longer, we should take the modules as an option and clear only those specified or exclude specified logs
Comment 1 Andrew Fuerste-Henry 2021-03-02 18:14:05 UTC 
+1 on this, libraries want some logs longer than others. Also, some logs are more feasible to keep longer. Granularity would be great.
Comment 2 Rebecca Coert 2021-03-04 21:33:13 UTC 
100% agree!  It's possible that management is going to request a report that requires us to look back at important data that it more than 180 days old.  It's impossible to predict what information you'll need to access & when.  Recently ran into this issue looking at holds placed/filled/etc. in July 2020.  The report is empty because the data's been scrubbed by the cron.  

I second Andrew's request for granularity.
Comment 3 Jeff Gaines 2021-03-04 21:42:39 UTC 
I would very much like more control over this setting. At the very least it should be a year's worth of data, but some smaller libraries might be able to hold multi-year logs. I support this request.
Comment 4 Nick Clemens (kidclamp) 2021-07-27 18:50:39 UTC 
Created attachment 123231 [details] [review]
Bug 18631: Add option to specify logs to preserve or delete in cleanup_database.pl

This adds two new parameters:
--log_module
--preserve_log

These can be repeated to include or exclude specific modules from the cleanup

To test:
0 - Apply patch
1 - Enable cataloging log and borrowers log
2 - Make some changes to borrowers and records
3 - run cleanup databse with --logs 0 --preserve_log=MEMBERS --preserve_log=CATALOGUING
4 - nothing is removed
5 - run cleanup databse with --logs 0 --log_module=MEMBERS
6 - the borrower logs are removed, the record changes remain
7 - run cleanup database with --logs 0
8 - record changes are removed (all other logs too)
Comment 5 Andrew Fuerste-Henry 2021-08-16 19:00:50 UTC 
Created attachment 123857 [details] [review]
Bug 18631: Add option to specify logs to preserve or delete in cleanup_database.pl

This adds two new parameters:
--log_module
--preserve_log

These can be repeated to include or exclude specific modules from the cleanup

To test:
0 - Apply patch
1 - Enable cataloging log and borrowers log
2 - Make some changes to borrowers and records
3 - run cleanup databse with --logs 0 --preserve_log=MEMBERS --preserve_log=CATALOGUING
4 - nothing is removed
5 - run cleanup databse with --logs 0 --log_module=MEMBERS
6 - the borrower logs are removed, the record changes remain
7 - run cleanup database with --logs 0
8 - record changes are removed (all other logs too)

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>

Signed-off-by: Rebecca Coert <rcoert@arlingtonva.us>
Comment 6 Martin Renvoize (ashimema) 2021-09-30 10:45:13 UTC 
Created attachment 125485 [details] [review]
Bug 18631: Add option to specify logs to preserve or delete in cleanup_database.pl

This adds two new parameters:
--log_module
--preserve_log

These can be repeated to include or exclude specific modules from the cleanup

To test:
0 - Apply patch
1 - Enable cataloging log and borrowers log
2 - Make some changes to borrowers and records
3 - run cleanup databse with --logs 0 --preserve_log=MEMBERS --preserve_log=CATALOGUING
4 - nothing is removed
5 - run cleanup databse with --logs 0 --log_module=MEMBERS
6 - the borrower logs are removed, the record changes remain
7 - run cleanup database with --logs 0
8 - record changes are removed (all other logs too)

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Rebecca Coert <rcoert@arlingtonva.us>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 7 Martin Renvoize (ashimema) 2021-09-30 10:46:18 UTC 
Nice improvement... I fixed up the patch to drop the unintended Koha/XSLT/Base.pm debugging.. they were clearly an errant part of the commit. ;)

Passing QA
Comment 8 Jonathan Druart 2021-10-07 09:12:04 UTC 
Created attachment 125864 [details] [review]
Bug 18631: Use - instead of _ for options

To match existing options
Comment 9 Jonathan Druart 2021-10-07 09:20:17 UTC 
Pushed to master for 21.11, thanks to everybody involved!