Bug 10955 - Add ability to skip deletions in zebraqueue
Summary: Add ability to skip deletions in zebraqueue
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Command-line Utilities (show other bugs)
Version: Main
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Kyle M Hall
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-09-26 16:47 UTC by Kyle M Hall
Modified: 2015-06-04 23:30 UTC (History)
4 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Small patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
Bug 10955 - Add ability to skip deletions in zebraqueue (5.78 KB, patch)
2013-09-26 16:50 UTC, Kyle M Hall
Details | Diff | Splinter Review
Fast Add framework sample (74.47 KB, application/xml)
2013-10-21 22:12 UTC, Elaine Bradtke
Details
Bug 10955 - Add ability to skip deletions in zebraqueue (4.74 KB, patch)
2014-01-15 16:42 UTC, Jonathan Druart
Details | Diff | Splinter Review
[PASSED QA] Bug 10955 - Add ability to skip deletions in zebraqueue (4.88 KB, patch)
2014-01-25 11:21 UTC, Katrin Fischer
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Kyle M Hall 2013-09-26 16:47:04 UTC
It seems that record deletions can cause extreme slowdowns for Koha installations with extremely large numbers of records. It would be helpful to be able to skip record deletions when processing the zebraqueue with rebuild_zebra.pl so the deletions can be processed with a lower frequency.
Comment 1 Kyle M Hall 2013-09-26 16:50:09 UTC Comment hidden (obsolete)
Comment 2 I'm just a bot 2013-09-29 06:22:09 UTC
Patch applied cleanly, go forth and signoff
Comment 3 Elaine Bradtke 2013-10-21 22:12:02 UTC Comment hidden (obsolete)
Comment 4 Elaine Bradtke 2013-10-21 22:16:08 UTC
Sorry, gremlins - that attachment went to the wrong bug!
Comment 5 Elaine Bradtke 2013-10-21 22:24:42 UTC
Comment on attachment 22204 [details]
Fast Add framework sample

this was attached to the wrong bug!
Comment 6 I'm just a bot 2013-10-30 07:43:03 UTC
Patch applied cleanly, go forth and signoff
Comment 7 Jonathan Druart 2014-01-15 16:42:07 UTC Comment hidden (obsolete)
Comment 8 Katrin Fischer 2014-01-25 11:21:42 UTC
Created attachment 24730 [details] [review]
[PASSED QA] Bug 10955 - Add ability to skip deletions in zebraqueue

It seems that record deletions can cause extreme slowdowns for Koha
installations with extremely large numbers of records. It would be
helpful to be able to skip record deletions when processing the
zebraqueue with rebuild_zebra.pl so the deletions can be processed with
a lower frequency.

Test Plan:
1) Disable any zebra indexing cronjobs you may have
2) Delete a record
3) Note the operation recordDelete in the zebraqueue table having done = 0
4) Run misc/migration_tools/rebuild_zebra.pl -b -z --skip-deletes
5) Note the delete still has done = 0
6) Run misc/migration_tools/rebuild_zebra.pl -b -z
7) Note the delete now has done = 1

Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Passes all tests and QA script.
Also tested for authorities, no problems found.
Comment 9 Galen Charlton 2014-03-10 18:47:50 UTC
Pushed to master.  Thanks, Kyle!

If the underlying issue in zebraidx never gets resolved, I think a better course for us in the long run is to introduce a soft-deletion mechanism -- basically, something like the current OPAC suppression mechanism, but which marks records as being logically deleted.
Comment 10 Fridolin Somers 2014-05-27 14:27:37 UTC
Pushed to 3.14.x, will be in 3.14.07