Bug 37987

Summary: Downloading SQL report in .tab format is slow
Product: Koha Reporter: Emmi Takkinen <emmi.takkinen>
Component: ReportsAssignee: Emmi Takkinen <emmi.takkinen>
Status: Needs documenting --- QA Contact: Paul Derscheid <paul.derscheid>
Severity: minor    
Priority: P5 - low CC: aleisha, fridolin.somers, lucas, ovezina, paul.derscheid
Version: Main   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
24.11.00,24.05.06
Circulation function:
Bug Depends on: 5920    
Bug Blocks:    
Attachments: Bug 37987: Speed up downloading reports in .tab format
Bug 37987: Speed up downloading reports in .tab format
Bug 37987: Speed up downloading reports in .tab format

Description Emmi Takkinen 2024-09-23 12:00:58 UTC
After changes made in bug 37382, downloading a SQL report has become significantly slower. I have a simple report to select first 4000 items from database. In version 23.11 it takes 1.27 seconds to download them into .tab file. On current master downloading same report takes 20 seconds.
Comment 1 Emmi Takkinen 2024-09-24 12:13:15 UTC
Created attachment 171924 [details] [review]
Bug 37987: Speed up downloading reports in .tab format

Downloading SQL report in .tab format was significantly
slower after changes made in bug 37382. Apparently using
HTML::Scrubber to redeclare content variable after
mapping it is slow. Instead sending whole map function
as parameter to module speeds up downloading to couple
of seconds as it was before.

To test:
1. Find or create report that returns several thousand rows
(e.g. SELECT * FROM items LIMIT 4000).
2. Run report and attempt to download it as .tab file.
=> Notice that this takes a lot of time (around 20 seconds).
3. Apply this patch.
4. Download report again.
=> Dowloading should now take couple second.
=> Confirm that the content of .tab file still contains table
data.

Sponsored-by: Koha-Suomi Oy
Comment 2 Olivier Vezina 2024-09-26 15:16:10 UTC
Created attachment 171982 [details] [review]
Bug 37987: Speed up downloading reports in .tab format

Downloading SQL report in .tab format was significantly
slower after changes made in bug 37382. Apparently using
HTML::Scrubber to redeclare content variable after
mapping it is slow. Instead sending whole map function
as parameter to module speeds up downloading to couple
of seconds as it was before.

To test:
1. Find or create report that returns several thousand rows
(e.g. SELECT * FROM items LIMIT 4000).
2. Run report and attempt to download it as .tab file.
=> Notice that this takes a lot of time (around 20 seconds).
3. Apply this patch.
4. Download report again.
=> Dowloading should now take couple second.
=> Confirm that the content of .tab file still contains table
data.

Sponsored-by: Koha-Suomi Oy
Comment 3 Olivier Vezina 2024-09-26 15:43:13 UTC
Signed-off just forgot to add -s to my commit
Comment 4 Paul Derscheid 2024-11-07 10:34:36 UTC
Created attachment 174108 [details] [review]
Bug 37987: Speed up downloading reports in .tab format

Downloading SQL report in .tab format was significantly
slower after changes made in bug 37382. Apparently using
HTML::Scrubber to redeclare content variable after
mapping it is slow. Instead sending whole map function
as parameter to module speeds up downloading to couple
of seconds as it was before.

To test:
1. Find or create report that returns several thousand rows
(e.g. SELECT * FROM items LIMIT 4000).
2. Run report and attempt to download it as .tab file.
=> Notice that this takes a lot of time (around 20 seconds).
3. Apply this patch.
4. Download report again.
=> Dowloading should now take couple second.
=> Confirm that the content of .tab file still contains table
data.

Sponsored-by: Koha-Suomi Oy
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Comment 5 Paul Derscheid 2024-11-07 10:35:11 UTC
Such a simple yet effective performance improvement! Great work.
Comment 6 Katrin Fischer 2024-11-08 10:24:52 UTC
Pushed for 24.11!

Well done everyone, thank you!
Comment 7 Lucas Gass (lukeg) 2024-11-15 16:16:32 UTC
Backported to 24.05.x for upcoming 24.05.06
Comment 8 Fridolin Somers 2024-12-02 08:43:05 UTC
Depends on Bug 5920 not in 23.11.x