Bug 37987 - Downloading SQL report in .tab format is slow
Summary: Downloading SQL report in .tab format is slow
Status: RESOLVED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Reports (show other bugs)
Version: Main
Hardware: All All
: P5 - low minor
Assignee: Emmi Takkinen
QA Contact: Paul Derscheid
URL:
Keywords:
Depends on: 5920
Blocks:
  Show dependency treegraph
 
Reported: 2024-09-23 12:00 UTC by Emmi Takkinen
Modified: 2025-04-18 19:51 UTC (History)
6 users (show)

See Also:
GIT URL:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
24.11.00,24.05.06
Circulation function:


Attachments
Bug 37987: Speed up downloading reports in .tab format (1.73 KB, patch)
2024-09-24 12:13 UTC, Emmi Takkinen
Details | Diff | Splinter Review
Bug 37987: Speed up downloading reports in .tab format (1.73 KB, patch)
2024-09-26 15:16 UTC, Olivier Vezina
Details | Diff | Splinter Review
Bug 37987: Speed up downloading reports in .tab format (1.79 KB, patch)
2024-11-07 10:34 UTC, Paul Derscheid
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Emmi Takkinen 2024-09-23 12:00:58 UTC
After changes made in bug 37382, downloading a SQL report has become significantly slower. I have a simple report to select first 4000 items from database. In version 23.11 it takes 1.27 seconds to download them into .tab file. On current master downloading same report takes 20 seconds.
Comment 1 Emmi Takkinen 2024-09-24 12:13:15 UTC
Created attachment 171924 [details] [review]
Bug 37987: Speed up downloading reports in .tab format

Downloading SQL report in .tab format was significantly
slower after changes made in bug 37382. Apparently using
HTML::Scrubber to redeclare content variable after
mapping it is slow. Instead sending whole map function
as parameter to module speeds up downloading to couple
of seconds as it was before.

To test:
1. Find or create report that returns several thousand rows
(e.g. SELECT * FROM items LIMIT 4000).
2. Run report and attempt to download it as .tab file.
=> Notice that this takes a lot of time (around 20 seconds).
3. Apply this patch.
4. Download report again.
=> Dowloading should now take couple second.
=> Confirm that the content of .tab file still contains table
data.

Sponsored-by: Koha-Suomi Oy
Comment 2 Olivier Vezina 2024-09-26 15:16:10 UTC
Created attachment 171982 [details] [review]
Bug 37987: Speed up downloading reports in .tab format

Downloading SQL report in .tab format was significantly
slower after changes made in bug 37382. Apparently using
HTML::Scrubber to redeclare content variable after
mapping it is slow. Instead sending whole map function
as parameter to module speeds up downloading to couple
of seconds as it was before.

To test:
1. Find or create report that returns several thousand rows
(e.g. SELECT * FROM items LIMIT 4000).
2. Run report and attempt to download it as .tab file.
=> Notice that this takes a lot of time (around 20 seconds).
3. Apply this patch.
4. Download report again.
=> Dowloading should now take couple second.
=> Confirm that the content of .tab file still contains table
data.

Sponsored-by: Koha-Suomi Oy
Comment 3 Olivier Vezina 2024-09-26 15:43:13 UTC
Signed-off just forgot to add -s to my commit
Comment 4 Paul Derscheid 2024-11-07 10:34:36 UTC
Created attachment 174108 [details] [review]
Bug 37987: Speed up downloading reports in .tab format

Downloading SQL report in .tab format was significantly
slower after changes made in bug 37382. Apparently using
HTML::Scrubber to redeclare content variable after
mapping it is slow. Instead sending whole map function
as parameter to module speeds up downloading to couple
of seconds as it was before.

To test:
1. Find or create report that returns several thousand rows
(e.g. SELECT * FROM items LIMIT 4000).
2. Run report and attempt to download it as .tab file.
=> Notice that this takes a lot of time (around 20 seconds).
3. Apply this patch.
4. Download report again.
=> Dowloading should now take couple second.
=> Confirm that the content of .tab file still contains table
data.

Sponsored-by: Koha-Suomi Oy
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Comment 5 Paul Derscheid 2024-11-07 10:35:11 UTC
Such a simple yet effective performance improvement! Great work.
Comment 6 Katrin Fischer 2024-11-08 10:24:52 UTC
Pushed for 24.11!

Well done everyone, thank you!
Comment 7 Lucas Gass (lukeg) 2024-11-15 16:16:32 UTC
Backported to 24.05.x for upcoming 24.05.06
Comment 8 Fridolin Somers 2024-12-02 08:43:05 UTC
Depends on Bug 5920 not in 23.11.x
Comment 9 Caroline Cyr La Rose 2025-04-18 19:51:39 UTC
Big fix, nothing to add/edit in the manual.