Bug 42028

Summary: DB backup improvements: use ZSTD compression and tweak default settings
Product: Koha Reporter: Michał <schodkowy.omegi-0r>
Component: Architecture, internals, and plumbingAssignee: Michał <schodkowy.omegi-0r>
Status: Needs Signoff --- QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: januszop, schodkowy.omegi-0r
Version: Main   
Hardware: All   
OS: All   
GIT URL: Initiative type: ---
Sponsorship status: --- Comma delimited list of Sponsors:
Crowdfunding goal: 0 Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:
Attachments: [PATCH 1/3] Bug 42028: use ZTSD instead of GZIP for koha-dump
[PATCH 2/3] Bug 42028: tweak default koha-run-backups settings
[PATCH 3/3] Bug 42028: set process niceness for koha-dump

Description Michał 2026-03-09 14:29:32 UTC

    
Comment 1 Michał 2026-03-09 14:53:52 UTC
Created attachment 195003 [details] [review]
[PATCH 1/3] Bug 42028: use ZTSD instead of GZIP for koha-dump

[PATCH 1/3] Bug 42028: use ZTSD instead of GZIP for koha-dump

Notes:

1. By default ZSTD is much faster, while also providing smaller files

2. In koha-run-backups I've reorganized the deletion to be more readable and support both old and new extension. The instance name is properly escaped juuust in case, and a single regex is used. Mtime option was simplified, it works the same way while being more lean. These shell expansions were tested in both bash and dash, so /bin/sh shebang should be fine to remain. Even though --days must be greater than 0, if someone removed that check, `-mtime +-1` would still work as expected, amazingly.

---

Benchmarking:

# time koha-dump biblioteka
Dumping Koha site biblioteka:
* DB to /var/spool/koha/biblioteka/biblioteka-2026-03-09.sql.gz
* configs, logs to /var/spool/koha/biblioteka/biblioteka-2026-03-09.tar.gz
Done.

real    1m13.647s
user    1m22.566s
sys     0m4.807s

# time koha-dump biblioteka
Dumping Koha site biblioteka:
* DB to /var/spool/koha/biblioteka/biblioteka-2026-03-09.sql.zst
* configs, logs to /var/spool/koha/biblioteka/biblioteka-2026-03-09.tar.zst
Done.

real    0m15.590s
user    0m22.709s
sys     0m5.305s

# ls -lh /var/spool/koha/biblioteka/ | grep -F '2026-03-09.'
-rw-r----- 1 root biblioteka-koha 111M Mar  9 15:20 biblioteka-2026-03-09.sql.gz
-rw-r----- 1 root biblioteka-koha  99M Mar  9 15:23 biblioteka-2026-03-09.sql.zst
-rw-r----- 1 root biblioteka-koha 949M Mar  9 15:21 biblioteka-2026-03-09.tar.gz
-rw-r----- 1 root biblioteka-koha 937M Mar  9 15:23 biblioteka-2026-03-09.tar.zst
Comment 2 Michał 2026-03-09 14:54:25 UTC
Created attachment 195004 [details] [review]
[PATCH 2/3] Bug 42028: tweak default koha-run-backups settings

[PATCH 2/3] Bug 42028: tweak default koha-run-backups settings

"Configuration" tar file should not be so outrageously big (~10x bigger than the database, what in the world!).

The biggest offender is the Zebra index files. These should not be backed up at all, instead regenerate them after restoring backup. There's no consistency between SQL and index dumps, they're not atomic, so restoring them like that is unsafe anyway. In fact, ideally that option should be removed completely so that nobody can shoot their foot off.

Logs these days can also grow quite big with all these crazy AI crawlers out there nowadays and default retention settings. I don't think making N copies of the very same logs on the server, which are meant to be ultimately ephemeral anyways, is really beneficial...

And now that we shrunk the needed storage by ~10x (YMMV), bump default 2 days to 7, making it much more likely you'll have old enough point of reference in case anything goes wrong with your Koha, before it gets nuked forever.
Comment 3 Michał 2026-03-09 14:54:50 UTC
Created attachment 195005 [details] [review]
[PATCH 3/3] Bug 42028: set process niceness for koha-dump

[PATCH 3/3] Bug 42028: set process niceness for koha-dump

Check niceness with htop, check ionice with: ionice -p `pidof zstd` (or replace zstd with mysqldump or tar)
Comment 4 Michał 2026-03-09 15:08:44 UTC
Test plan:

PATCH 1:

1. Before applying run `time koha-dump INSTANCE`

2. Apply patch

3. Run #1 again and compare the times, run `ls -lh /var/spool/koha/INSTANCE` and compare the file sizes for .gz and .zst

Optionally you may run `koha-run-backups --days 7 --exclude-indexes --exclude-logs --output /var/spool/koha` to make sure it works as expected, altering file name and date with `touch -d` or so, to make sure the --days option still works as expected and old files are getting removed.

PATCH 2:

1. Run old cron command: `koha-run-backups --days 2 --output /var/spool/koha`

2. Look at the generated file size of .tar.zst in /var/spool/koha/INSTANCE

3. Run new command: `koha-run-backups --days 7 --exclude-indexes --exclude-logs --output /var/spool/koha`

4. Compare the sizes. Note that production installations with sizable Zebra index and log files will only yield the proper difference effect.

PATCH 3:

Run `koha-dump INSTANCE` and in another terminal use a tool like htop and you can sort by "NI" column to make sure koha-dump and its children processes have value of 10. You may also check ionice with: ionice -p `pidof zstd` (or replace zstd with mysqldump or tar). Be quick, since zstd backup finished much more quickly, you'd need to check these values before it finishes.

The error messages for renice and ionice should not be shown at the top of your koha-dump terminal output (own process normally can adjust own values, and also these scripts are in sbin to only run as root...).