Bug 26424 - Better performance of svc/checkouts
Summary: Better performance of svc/checkouts
Status: Pushed to stable
Alias: None
Product: Koha
Classification: Unclassified
Component: Circulation (show other bugs)
Version: master
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Fridolin Somers
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-09-10 13:11 UTC by Fridolin Somers
Modified: 2020-10-27 03:53 UTC (History)
7 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Small patch
Who signed the patch off:
Manual merge request:
Text to go in the release notes:
Version(s) released in:
20.11.00, 20.05.05


Attachments
Bug 26424: Better performance of svc/checkouts (4.81 KB, patch)
2020-09-10 13:26 UTC, Fridolin Somers
Details | Diff | Splinter Review
Bug 26424: Better performance of svc/checkouts (5.03 KB, patch)
2020-09-10 13:52 UTC, Fridolin Somers
Details | Diff | Splinter Review
Bug 26424: Better performance of svc/checkouts (5.20 KB, patch)
2020-09-11 07:08 UTC, Fridolin Somers
Details | Diff | Splinter Review
Bug 26424: Better performance of svc/checkouts (5.27 KB, patch)
2020-09-16 14:13 UTC, Martin Renvoize
Details | Diff | Splinter Review
Bug 26424: Better performance of svc/checkouts (5.25 KB, patch)
2020-09-18 12:44 UTC, Fridolin Somers
Details | Diff | Splinter Review
Bug 26424: Better performance of svc/checkouts (5.32 KB, patch)
2020-09-18 12:46 UTC, Fridolin Somers
Details | Diff | Splinter Review
Bug 26424: Better performance of svc/checkouts (5.37 KB, patch)
2020-09-20 17:35 UTC, Katrin Fischer
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Fridolin Somers 2020-09-10 13:11:55 UTC
Ajax script svc/checkouts display checkouts of a patron.
For each item, it fetches a Koha::ItemType object and a Koha::AuthorisedValues object for location,ccode,lost and damaged.

For performance on huge number of checkouts :
Item types should be fetch once before the loop.
authorised values should call Koha::AuthorisedValues->get_description_by_koha_field because it uses a cache.
Comment 1 Fridolin Somers 2020-09-10 13:26:16 UTC Comment hidden (obsolete)
Comment 2 Fridolin Somers 2020-09-10 13:52:48 UTC
Created attachment 109851 [details] [review]
Bug 26424: Better performance of svc/checkouts

Ajax script svc/checkouts display checkouts of a patron.
For each item, it fetches a Koha::ItemType object and a Koha::AuthorisedValues object for location,ccode,lost and damaged.

For performance on huge number of checkouts :
Item types should be fetch once before the loop.
authorised values should call Koha::AuthorisedValues->get_description_by_koha_field because it uses a cache.

I've tested with Plack :
Without patch :
100 checkouts = 6 seconds
1000 checkouts = 60 seconds
With patch :
100 checkouts = 5 seconds
1000 checkouts = 44 seconds

Test plan :
1) Dont apply patch
2) Use sql to define some items lost and damaged
3) Look at checkouts table on a patron with a lot of checkouts
4) Apply patch
5) Look at checkouts table again
6) Check infos are the same : record level item type, item type, location, collection, lost, damaged
7) Check infos are the same in "Number of checkouts by item type"
Comment 3 Tomás Cohen Arazi 2020-09-10 14:17:34 UTC
It's ok, but we should use the REST API sooner than later. We can embed things, etc.
Comment 4 Fridolin Somers 2020-09-11 06:41:39 UTC
(In reply to Tomás Cohen Arazi from comment #3)
> It's ok, but we should use the REST API sooner than later. We can embed
> things, etc.

Yep indeed, but that will be a lot of work, this table is really complex.
Comment 5 Fridolin Somers 2020-09-11 07:07:31 UTC
Ah patch also changes the fact that authorised value categories are no longer hardcoded LOC,CCODE,LOST and DAMAGED, they depend on default framework.
Like is doing Bug 26323
Comment 6 Fridolin Somers 2020-09-11 07:08:17 UTC
Created attachment 109883 [details] [review]
Bug 26424: Better performance of svc/checkouts

Ajax script svc/checkouts display checkouts of a patron.
For each item, it fetches a Koha::ItemType object and a Koha::AuthorisedValues object for location,ccode,lost and damaged.

For performance on huge number of checkouts :
Item types should be fetch once before the loop.
authorised values should call Koha::AuthorisedValues->get_description_by_koha_field because it uses a cache.

I've tested with Plack :
Without patch :
100 checkouts = 6 seconds
1000 checkouts = 60 seconds
With patch :
100 checkouts = 5 seconds
1000 checkouts = 44 seconds

Patch also changes the fact that authorised value categories are no longer hardcoded LOC,CCODE,LOST and DAMAGED, they depend on default framework.
Like is doing Bug 26323.

Test plan :
1) Dont apply patch
2) Use sql to define some items lost and damaged
3) Look at checkouts table on a patron with a lot of checkouts
4) Apply patch
5) Look at checkouts table again
6) Check infos are the same : record level item type, item type, location, collection, lost, damaged
7) Check infos are the same in "Number of checkouts by item type"
Comment 7 Martin Renvoize 2020-09-16 14:13:08 UTC
Created attachment 110209 [details] [review]
Bug 26424: Better performance of svc/checkouts

Ajax script svc/checkouts display checkouts of a patron.
For each item, it fetches a Koha::ItemType object and a Koha::AuthorisedValues object for location,ccode,lost and damaged.

For performance on huge number of checkouts :
Item types should be fetch once before the loop.
authorised values should call Koha::AuthorisedValues->get_description_by_koha_field because it uses a cache.

I've tested with Plack :
Without patch :
100 checkouts = 6 seconds
1000 checkouts = 60 seconds
With patch :
100 checkouts = 5 seconds
1000 checkouts = 44 seconds

Patch also changes the fact that authorised value categories are no longer hardcoded LOC,CCODE,LOST and DAMAGED, they depend on default framework.
Like is doing Bug 26323.

Test plan :
1) Dont apply patch
2) Use sql to define some items lost and damaged
3) Look at checkouts table on a patron with a lot of checkouts
4) Apply patch
5) Look at checkouts table again
6) Check infos are the same : record level item type, item type, location, collection, lost, damaged
7) Check infos are the same in "Number of checkouts by item type"

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 8 Martin Renvoize 2020-09-16 14:13:32 UTC
Clear and sensible improvement works as expected.

Signing off
Comment 9 Katrin Fischer 2020-09-17 21:12:12 UTC
Please check the fail from the QA script:

 FAIL	svc/checkouts
   FAIL	  critic
		# Variables::ProhibitConditionalDeclarations: Got 2 violation(s).
Comment 10 Fridolin Somers 2020-09-18 12:44:01 UTC
Created attachment 110378 [details] [review]
Bug 26424: Better performance of svc/checkouts

Ajax script svc/checkouts display checkouts of a patron.
For each item, it fetches a Koha::ItemType object and a Koha::AuthorisedValues object for location,ccode,lost and damaged.

For performance on huge number of checkouts :
Item types should be fetch once before the loop.
authorised values should call Koha::AuthorisedValues->get_description_by_koha_field because it uses a cache.

I've tested with Plack :
Without patch :
100 checkouts = 6 seconds
1000 checkouts = 60 seconds
With patch :
100 checkouts = 5 seconds
1000 checkouts = 44 seconds

Patch also changes the fact that authorised value categories are no longer hardcoded LOC,CCODE,LOST and DAMAGED, they depend on default framework.
Like is doing Bug 26323.

Test plan :
1) Dont apply patch
2) Use sql to define some items lost and damaged
3) Look at checkouts table on a patron with a lot of checkouts
4) Apply patch
5) Look at checkouts table again
6) Check infos are the same : record level item type, item type, location, collection, lost, damaged
7) Check infos are the same in "Number of checkouts by item type"
Comment 11 Fridolin Somers 2020-09-18 12:45:01 UTC
(In reply to Katrin Fischer from comment #9)
> Please check the fail from the QA script:
> 
>  FAIL	svc/checkouts
>    FAIL	  critic
> 		# Variables::ProhibitConditionalDeclarations: Got 2 violation(s).

Oh indeed i see.

Its a minor change :

-    my $type_for_stat = Koha::ItemTypes->find( $item_level_itypes ? $c->{itype} : $c->{itemtype} );
-    my $itemtype = Koha::ItemTypes->find( $c->{itype} );
-    my $recordtype = Koha::ItemTypes->find( $c->{itemtype} );
+    my ( $itemtype, $recordtype, $type_for_stat );
+    $itemtype      = $itemtypes->{ $c->{itype} }    if $c->{itype};
+    $recordtype    = $itemtypes->{ $c->{itemtype} } if $c->{itemtype};
+    $type_for_stat = $item_level_itypes ? $itemtype : $recordtype;

I replaced the patch
Comment 12 Fridolin Somers 2020-09-18 12:46:15 UTC
Created attachment 110379 [details] [review]
Bug 26424: Better performance of svc/checkouts

Ajax script svc/checkouts display checkouts of a patron.
For each item, it fetches a Koha::ItemType object and a Koha::AuthorisedValues object for location,ccode,lost and damaged.

For performance on huge number of checkouts :
Item types should be fetch once before the loop.
authorised values should call Koha::AuthorisedValues->get_description_by_koha_field because it uses a cache.

I've tested with Plack :
Without patch :
100 checkouts = 6 seconds
1000 checkouts = 60 seconds
With patch :
100 checkouts = 5 seconds
1000 checkouts = 44 seconds

Patch also changes the fact that authorised value categories are no longer hardcoded LOC,CCODE,LOST and DAMAGED, they depend on default framework.
Like is doing Bug 26323.

Test plan :
1) Dont apply patch
2) Use sql to define some items lost and damaged
3) Look at checkouts table on a patron with a lot of checkouts
4) Apply patch
5) Look at checkouts table again
6) Check infos are the same : record level item type, item type, location, collection, lost, damaged
7) Check infos are the same in "Number of checkouts by item type"

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 13 Katrin Fischer 2020-09-20 17:35:00 UTC
Created attachment 110429 [details] [review]
Bug 26424: Better performance of svc/checkouts

Ajax script svc/checkouts display checkouts of a patron.
For each item, it fetches a Koha::ItemType object and a Koha::AuthorisedValues object for location,ccode,lost and damaged.

For performance on huge number of checkouts :
Item types should be fetch once before the loop.
authorised values should call Koha::AuthorisedValues->get_description_by_koha_field because it uses a cache.

I've tested with Plack :
Without patch :
100 checkouts = 6 seconds
1000 checkouts = 60 seconds
With patch :
100 checkouts = 5 seconds
1000 checkouts = 44 seconds

Patch also changes the fact that authorised value categories are no longer hardcoded LOC,CCODE,LOST and DAMAGED, they depend on default framework.
Like is doing Bug 26323.

Test plan :
1) Dont apply patch
2) Use sql to define some items lost and damaged
3) Look at checkouts table on a patron with a lot of checkouts
4) Apply patch
5) Look at checkouts table again
6) Check infos are the same : record level item type, item type, location, collection, lost, damaged
7) Check infos are the same in "Number of checkouts by item type"

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 14 Jonathan Druart 2020-09-28 14:12:12 UTC
Pushed to master for 20.11, thanks to everybody involved!
Comment 15 Lucas Gass 2020-10-20 13:47:37 UTC
backported to 20.05.x for 20.05.05
Comment 16 Aleisha Amohia 2020-10-27 03:53:13 UTC
enhancement, not backported to 19.11.x