Bug 14532

Summary: koha-dump should provide a way to exclude Zebra indexes
Product: Koha Reporter: Tomás Cohen Arazi (tcohen) <tomascohen>
Component: Command-line UtilitiesAssignee: Tomás Cohen Arazi (tcohen) <tomascohen>
Status: CLOSED FIXED QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: barton, brendan, dcook, jonathan.druart, josef.moravec, mtj, robin
Version: Main   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: Small patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:
Bug Depends on:    
Bug Blocks: 21111    
Attachments: Bug 14532: Add --include-indexes switch to koha-dump
Bug 14532: Add --exclude-indexes switch to koha-dump
Bug 14532 - fix syntax error in script
Bug 14532: fixed the "empty member" problem
Bug 14532: fixed the "empty member" problem
Bug 14532: (followup) Use tar's --exclude for simplicity
Bug 14532: (followup) Use tar's --exclude for simplicity
Bug 14532: Add --exclude-indexes switch to koha-dump
Bug 14532 - fix syntax error in script
Bug 14532: fixed the "empty member" problem
Bug 14532: (followup) Use tar's --exclude for simplicity

Description Tomás Cohen Arazi (tcohen) 2015-07-14 14:42:45 UTC
For regular use, it is not that important to save indexes every time the backup utility is run. We should add an option switch (like --indexes) so it needs to be explicitly requested.
It would mean a one-line change on the cron file for people really interested.
Comment 1 Robin Sheat 2015-07-14 23:33:33 UTC
I'm OK with that. The principle of backing up the indices is that it means a restore will have you up and running in next to no time. But making it optional is fine.
Comment 2 Tomás Cohen Arazi (tcohen) 2015-07-20 15:24:12 UTC Comment hidden (obsolete)
Comment 3 Tomás Cohen Arazi (tcohen) 2015-11-04 23:18:23 UTC Comment hidden (obsolete)
Comment 4 Robin Sheat 2015-11-10 02:47:44 UTC
$ ./koha-dump test
./koha-dump: 87: ./koha-dump: Syntax error: "fi" unexpected (expecting "then")
Comment 5 Robin Sheat 2015-11-10 02:51:26 UTC
$ sudo ./koha-dump --exclude-indexes test
Dumping Koha site test:
* DB to /var/spool/koha/test/test-2015-11-10.sql.gz
* configs, logs to /var/spool/koha/test/test-2015-11-10.tar.gz
tar: Substituting `.' for empty member name
tar: : Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
Comment 6 Robin Sheat 2015-11-10 03:18:35 UTC Comment hidden (obsolete)
Comment 7 Robin Sheat 2015-11-10 03:18:46 UTC Comment hidden (obsolete)
Comment 8 Robin Sheat 2015-11-10 03:21:47 UTC Comment hidden (obsolete)
Comment 9 Jonathan Druart 2015-11-10 10:12:49 UTC
Without the option, /var/lib/koha/$instance/plugins was dumped, with the option it's not.
Comment 10 Robin Sheat 2015-11-10 10:26:08 UTC
(In reply to Jonathan Druart from comment #9)
> Without the option, /var/lib/koha/$instance/plugins was dumped, with the
> option it's not.

Good point! Can we just do excludes on tar? That'd be safer.
Comment 11 Tomás Cohen Arazi (tcohen) 2016-01-21 13:52:44 UTC Comment hidden (obsolete)
Comment 12 Mason James 2016-01-26 01:55:55 UTC
(In reply to Tomás Cohen Arazi from comment #11)
> Created attachment 47117 [details] [review] [review]
> Bug 14532: (followup) Use tar's --exclude for simplicity
> 
> This patch creates an --exclude definition for the tar command
> so it is easier to stack excluded stuff. It does so for the
> authority and biblio indexes if --exclude-indexes is passed.
> 
> A side effect from this, is that uploads and plugins are still
> backed up, as Jonathan noted.
> 
> Regards

Hi folks, can this be switched to 'sign-off' now?
Comment 13 Tomás Cohen Arazi (tcohen) 2016-01-26 02:29:31 UTC
(In reply to Mason James from comment #12)
> (In reply to Tomás Cohen Arazi from comment #11)
> > Created attachment 47117 [details] [review] [review] [review]
> > Bug 14532: (followup) Use tar's --exclude for simplicity
> > 
> > This patch creates an --exclude definition for the tar command
> > so it is easier to stack excluded stuff. It does so for the
> > authority and biblio indexes if --exclude-indexes is passed.
> > 
> > A side effect from this, is that uploads and plugins are still
> > backed up, as Jonathan noted.
> > 
> > Regards
> 
> Hi folks, can this be switched to 'sign-off' now?

Mason, when I was double-checking my followup, I figured it was not excluding things (using the same construct that worked on command-line), so i obsoleted it to avoid confusion, because I was leaving the office and wasn't sure when I was getting back to this. It is probably some typo from me, just didn't have the time that day.
Comment 14 Tomás Cohen Arazi (tcohen) 2016-01-26 14:59:50 UTC Comment hidden (obsolete)
Comment 15 Jonathan Druart 2016-01-27 14:23:26 UTC
Created attachment 47348 [details] [review]
Bug 14532: Add --exclude-indexes switch to koha-dump

This patch changes the default behaviour of koha-dump to make the inclusion
of Zebra indexes on the dump optional. It does so by introducing a new option
switch that allows to have the previous behaviour in place.

To test:
- Run
  $ koha-dump your_instance
- Save a copy of the dump files
- Apply the patch / extract the koha-dump script
- Run the new one:
  $ koha-dump your_instance
=> SUCCESS: Verify the contents of the dump are the same
   (i.e. it includes /var/lib/koha/your_instance)
- Run with the new switch:
  $ koha-dump --exclude-indexes your_instance
=> SUCCESS: The dump does not contain stuff from /var/lib/koha/your_instance
- Go through the rest of the new option switches
  -h | --help
  -q | --quiet
=> SUCCESS: They work as expected.
- Sign off :-D

Regards

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 16 Jonathan Druart 2016-01-27 14:23:30 UTC
Created attachment 47349 [details] [review]
Bug 14532 - fix syntax error in script

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 17 Jonathan Druart 2016-01-27 14:23:33 UTC
Created attachment 47350 [details] [review]
Bug 14532: fixed the "empty member" problem

PS: thanks to Olly Betts for knowing weird bash constuctions.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 18 Jonathan Druart 2016-01-27 14:23:37 UTC
Created attachment 47351 [details] [review]
Bug 14532: (followup) Use tar's --exclude for simplicity

This patch creates an --exclude definition for the tar command
so it is easier to stack excluded stuff. It does so for the
authority and biblio indexes if --exclude-indexes is passed.

A side effect from this, is that uploads and plugins are still
backed up, as Jonathan noted.

Regards

Edit: Fixed a weird quoting problem.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 19 Katrin Fischer 2016-01-27 23:49:55 UTC
I guess title should be 'should' - not shouldn or shouldn't? Changing it, please change back if wrong!
Comment 20 Brendan Gallagher 2016-03-02 04:45:30 UTC
Pushed to Master - Should be in the May 2016 Release. thanks!