Bug 25088 - The tool to bulk edit due dates proxy error times out with large data sets
Summary: The tool to bulk edit due dates proxy error times out with large data sets
Status: RESOLVED DUPLICATE of bug 25101
Alias: None
Product: Koha
Classification: Unclassified
Component: Tools (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal (vote)
Assignee: Bugs List
QA Contact: Testopia
URL:
Keywords:
Depends on: 24846
Blocks:
  Show dependency treegraph
 
Reported: 2020-04-08 20:37 UTC by Margaret Hade
Modified: 2020-04-09 18:18 UTC (History)
3 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
Bug 25088: prevent too many datetime operations (4.03 KB, patch)
2020-04-09 10:37 UTC, Jonathan Druart
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Margaret Hade 2020-04-08 20:37:49 UTC
The libraries we support are really appreciative of the new Bug 24846 - "Add a tool to bulk edit due dates", but if a library has 20,000, 30,000, 50,000 issues that they need to use the tool on the user will encounter a proxy error and the page will time out. We've tested this problem trying to even load the second page that shows you a preview of the record to be updated if pushing forward the due date by a number of days is the chosen option. We've gotten the page to load with a hard due date chosen; it squeaked under the apache time out at 8 minutes 46 seconds to load. 

It would be great if the tool had a better way of handling large batches of data to be updated, because the current errors aren't inspiring much confidence. Libraries can break their batches into smaller groups, to a point, but we're finding that it's still easier for them to have the support team do a bulk due date update for them. 

I know other tools encounter this issue with large datasets, but at this point we don't have an approximate number of issues that is it 'safe' to update at one time. And if they've got 10,000 items at a branch, all checked out by the same category, they might not be able to make that grouping small enough to get the tool to work.
Comment 1 Katrin Fischer 2020-04-08 21:08:32 UTC
By linking to the initial bug this will be a little easier to spot!
Comment 2 Jonathan Druart 2020-04-09 10:35:11 UTC
There is not much we can do.

On my laptop, after super quick tests, ~1100 checkouts is processed in ~13s
I made it down to 12s with small improvements (datetime handling), but it won't bring significant changes.

If we want to display before the process, it will be time consuming.

With bug 22417 in mind we could imagine bigger batches (background processing).
Or we can provide command line script, but I guess we are loosing the idea of an easy tool that can be used by librarians.

You could try playing with the Apache timeout value, but that will need a sysop intervention.
Comment 3 Jonathan Druart 2020-04-09 10:37:34 UTC
Created attachment 102610 [details] [review]
Bug 25088: prevent too many datetime operations

dropping this now to not forget, but it's not super useful.

Note for myself: no sure about the removal of $new_hard_due_date->clone
in the sub
Comment 4 Kyle M Hall 2020-04-09 18:18:58 UTC

*** This bug has been marked as a duplicate of bug 25101 ***