Bug 11343 - The max number of items to process in a batch is hardcoded
Summary: The max number of items to process in a batch is hardcoded
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Tools (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal (vote)
Assignee: Jonathan Druart
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-12-05 14:15 UTC by Jonathan Druart
Modified: 2015-12-18 11:59 UTC (History)
4 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Small patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
Bug 11343: The max number of items to process in a batch is hardcoded (5.05 KB, patch)
2013-12-05 14:36 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 11343: The max number of items to process in a batch is hardcoded (5.67 KB, patch)
2013-12-11 18:25 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 11343: The max number of items to process in a batch is hardcoded (5.70 KB, patch)
2014-01-07 22:54 UTC, Biblibre Sandboxes
Details | Diff | Splinter Review
Bug 11343: The max number of items to process in a batch is hardcoded (5.76 KB, patch)
2014-01-10 14:42 UTC, Kyle M Hall
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Jonathan Druart 2013-12-05 14:15:52 UTC
The maximum number of item records to process in a batch is 1000. This value is hardcoded in the code.
It should be store in a syspref.
Comment 1 Jonathan Druart 2013-12-05 14:36:38 UTC Comment hidden (obsolete)
Comment 2 Galen Charlton 2013-12-11 18:15:05 UTC
This patch appears to be missing the creation of koha-tmpl/intranet-tmpl/prog/en/modules/admin/preferences/tools.pref.
Comment 3 Jonathan Druart 2013-12-11 18:25:01 UTC
Arg, thank for the catch!
Comment 4 Jonathan Druart 2013-12-11 18:25:59 UTC Comment hidden (obsolete)
Comment 5 Christopher Brannon 2013-12-27 17:13:28 UTC
Jonathan,
This patch is inconsistent with results.

- Set the pref to 2
- try to delete a batch of 3 items

This will not show the items, but you can still delete the items.

Can anyone confirm if this issue exists with the limit hard coded?  I don't have 1001 items to test if going past the hard coded limit would produce the same result.  This could be revealing a pre-existing issue.

Also, should there be a maximum limit to how much can be processed? It seems to me if you are batch loading 1000 items, and it is displaying all those items, it might cause the entire system to drag.  But I am just guessing.  And if someone were to set a limit higher than 1000, would that potentially be worse?  In theory, it might be a performance issue, but in reality, would there ever be a case where someone would batch process thousands of records and cripple Koha in the process?

Christopher
Comment 6 Jonathan Druart 2014-01-06 16:21:25 UTC
Christopher,
The patch is consistent with the test plan: you are allow to modify XXX items and to display XXX items on deletion.
It is the same behavior as before the patch.

But I agree with you, it is not consistent with the syspref description (the description says: number of items to process).
I don't have a quick and small patch to provide. What I know it is that the limit is hardcoded and it is not possible to modify more than 1000 items in a batch, which is a real limitation for some libraries.
Comment 7 Christopher Brannon 2014-01-07 22:47:05 UTC
I will go ahead and sign off on this then and file a new bug regarding the processing issue.

Christopher
Comment 8 Biblibre Sandboxes 2014-01-07 22:54:07 UTC
Patch tested with a sandbox, by Christopher Brannon <cbrannon@cdalibrary.org>
Comment 9 Biblibre Sandboxes 2014-01-07 22:54:26 UTC Comment hidden (obsolete)
Comment 10 Kyle M Hall 2014-01-10 14:42:31 UTC
Created attachment 24133 [details] [review]
Bug 11343: The max number of items to process in a batch is hardcoded

Until now, the maximum number of item records to process in a batch was
hardcoded to 1000.
This patch adds a syspref MaxItemsForBatch in order to allow to adapt
this value.

Test plan:
- set the pref to 2
- try to delete a batch of 3 items: they are not displayed
- try to modify a batch of 3 items: you are not allowed to do that
- set the pref to 1000 and try again. Now items are displayed and you
  are allow to modify them.

Signed-off-by: Christopher Brannon <cbrannon@cdalibrary.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 11 Galen Charlton 2014-03-10 19:22:48 UTC
Pushed to master.  Thanks, Jonathan!
Comment 12 Fridolin Somers 2014-05-27 14:34:32 UTC
Pushed to 3.14.x, will be in 3.14.07