Bug 26424

Summary: Better performance of svc/checkouts
Product: Koha Reporter: Fridolin Somers <fridolin.somers>
Component: CirculationAssignee: Fridolin Somers <fridolin.somers>
Status: CLOSED FIXED QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: aleisha, dcook, gmcharlt, kyle.m.hall, lucas, martin.renvoize, tomascohen
Version: Main   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26323
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26393
Change sponsored?: --- Patch complexity: Small patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
20.11.00, 20.05.05
Attachments: Bug 26424: Better performance of svc/checkouts
Bug 26424: Better performance of svc/checkouts
Bug 26424: Better performance of svc/checkouts
Bug 26424: Better performance of svc/checkouts
Bug 26424: Better performance of svc/checkouts
Bug 26424: Better performance of svc/checkouts
Bug 26424: Better performance of svc/checkouts

Description Fridolin Somers 2020-09-10 13:11:55 UTC
Ajax script svc/checkouts display checkouts of a patron.
For each item, it fetches a Koha::ItemType object and a Koha::AuthorisedValues object for location,ccode,lost and damaged.

For performance on huge number of checkouts :
Item types should be fetch once before the loop.
authorised values should call Koha::AuthorisedValues->get_description_by_koha_field because it uses a cache.
Comment 1 Fridolin Somers 2020-09-10 13:26:16 UTC Comment hidden (obsolete)
Comment 2 Fridolin Somers 2020-09-10 13:52:48 UTC
Created attachment 109851 [details] [review]
Bug 26424: Better performance of svc/checkouts

Ajax script svc/checkouts display checkouts of a patron.
For each item, it fetches a Koha::ItemType object and a Koha::AuthorisedValues object for location,ccode,lost and damaged.

For performance on huge number of checkouts :
Item types should be fetch once before the loop.
authorised values should call Koha::AuthorisedValues->get_description_by_koha_field because it uses a cache.

I've tested with Plack :
Without patch :
100 checkouts = 6 seconds
1000 checkouts = 60 seconds
With patch :
100 checkouts = 5 seconds
1000 checkouts = 44 seconds

Test plan :
1) Dont apply patch
2) Use sql to define some items lost and damaged
3) Look at checkouts table on a patron with a lot of checkouts
4) Apply patch
5) Look at checkouts table again
6) Check infos are the same : record level item type, item type, location, collection, lost, damaged
7) Check infos are the same in "Number of checkouts by item type"
Comment 3 Tomás Cohen Arazi 2020-09-10 14:17:34 UTC
It's ok, but we should use the REST API sooner than later. We can embed things, etc.
Comment 4 Fridolin Somers 2020-09-11 06:41:39 UTC
(In reply to Tomás Cohen Arazi from comment #3)
> It's ok, but we should use the REST API sooner than later. We can embed
> things, etc.

Yep indeed, but that will be a lot of work, this table is really complex.
Comment 5 Fridolin Somers 2020-09-11 07:07:31 UTC
Ah patch also changes the fact that authorised value categories are no longer hardcoded LOC,CCODE,LOST and DAMAGED, they depend on default framework.
Like is doing Bug 26323
Comment 6 Fridolin Somers 2020-09-11 07:08:17 UTC
Created attachment 109883 [details] [review]
Bug 26424: Better performance of svc/checkouts

Ajax script svc/checkouts display checkouts of a patron.
For each item, it fetches a Koha::ItemType object and a Koha::AuthorisedValues object for location,ccode,lost and damaged.

For performance on huge number of checkouts :
Item types should be fetch once before the loop.
authorised values should call Koha::AuthorisedValues->get_description_by_koha_field because it uses a cache.

I've tested with Plack :
Without patch :
100 checkouts = 6 seconds
1000 checkouts = 60 seconds
With patch :
100 checkouts = 5 seconds
1000 checkouts = 44 seconds

Patch also changes the fact that authorised value categories are no longer hardcoded LOC,CCODE,LOST and DAMAGED, they depend on default framework.
Like is doing Bug 26323.

Test plan :
1) Dont apply patch
2) Use sql to define some items lost and damaged
3) Look at checkouts table on a patron with a lot of checkouts
4) Apply patch
5) Look at checkouts table again
6) Check infos are the same : record level item type, item type, location, collection, lost, damaged
7) Check infos are the same in "Number of checkouts by item type"
Comment 7 Martin Renvoize 2020-09-16 14:13:08 UTC
Created attachment 110209 [details] [review]
Bug 26424: Better performance of svc/checkouts

Ajax script svc/checkouts display checkouts of a patron.
For each item, it fetches a Koha::ItemType object and a Koha::AuthorisedValues object for location,ccode,lost and damaged.

For performance on huge number of checkouts :
Item types should be fetch once before the loop.
authorised values should call Koha::AuthorisedValues->get_description_by_koha_field because it uses a cache.

I've tested with Plack :
Without patch :
100 checkouts = 6 seconds
1000 checkouts = 60 seconds
With patch :
100 checkouts = 5 seconds
1000 checkouts = 44 seconds

Patch also changes the fact that authorised value categories are no longer hardcoded LOC,CCODE,LOST and DAMAGED, they depend on default framework.
Like is doing Bug 26323.

Test plan :
1) Dont apply patch
2) Use sql to define some items lost and damaged
3) Look at checkouts table on a patron with a lot of checkouts
4) Apply patch
5) Look at checkouts table again
6) Check infos are the same : record level item type, item type, location, collection, lost, damaged
7) Check infos are the same in "Number of checkouts by item type"

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 8 Martin Renvoize 2020-09-16 14:13:32 UTC
Clear and sensible improvement works as expected.

Signing off
Comment 9 Katrin Fischer 2020-09-17 21:12:12 UTC
Please check the fail from the QA script:

 FAIL	svc/checkouts
   FAIL	  critic
		# Variables::ProhibitConditionalDeclarations: Got 2 violation(s).
Comment 10 Fridolin Somers 2020-09-18 12:44:01 UTC
Created attachment 110378 [details] [review]
Bug 26424: Better performance of svc/checkouts

Ajax script svc/checkouts display checkouts of a patron.
For each item, it fetches a Koha::ItemType object and a Koha::AuthorisedValues object for location,ccode,lost and damaged.

For performance on huge number of checkouts :
Item types should be fetch once before the loop.
authorised values should call Koha::AuthorisedValues->get_description_by_koha_field because it uses a cache.

I've tested with Plack :
Without patch :
100 checkouts = 6 seconds
1000 checkouts = 60 seconds
With patch :
100 checkouts = 5 seconds
1000 checkouts = 44 seconds

Patch also changes the fact that authorised value categories are no longer hardcoded LOC,CCODE,LOST and DAMAGED, they depend on default framework.
Like is doing Bug 26323.

Test plan :
1) Dont apply patch
2) Use sql to define some items lost and damaged
3) Look at checkouts table on a patron with a lot of checkouts
4) Apply patch
5) Look at checkouts table again
6) Check infos are the same : record level item type, item type, location, collection, lost, damaged
7) Check infos are the same in "Number of checkouts by item type"
Comment 11 Fridolin Somers 2020-09-18 12:45:01 UTC
(In reply to Katrin Fischer from comment #9)
> Please check the fail from the QA script:
> 
>  FAIL	svc/checkouts
>    FAIL	  critic
> 		# Variables::ProhibitConditionalDeclarations: Got 2 violation(s).

Oh indeed i see.

Its a minor change :

-    my $type_for_stat = Koha::ItemTypes->find( $item_level_itypes ? $c->{itype} : $c->{itemtype} );
-    my $itemtype = Koha::ItemTypes->find( $c->{itype} );
-    my $recordtype = Koha::ItemTypes->find( $c->{itemtype} );
+    my ( $itemtype, $recordtype, $type_for_stat );
+    $itemtype      = $itemtypes->{ $c->{itype} }    if $c->{itype};
+    $recordtype    = $itemtypes->{ $c->{itemtype} } if $c->{itemtype};
+    $type_for_stat = $item_level_itypes ? $itemtype : $recordtype;

I replaced the patch
Comment 12 Fridolin Somers 2020-09-18 12:46:15 UTC
Created attachment 110379 [details] [review]
Bug 26424: Better performance of svc/checkouts

Ajax script svc/checkouts display checkouts of a patron.
For each item, it fetches a Koha::ItemType object and a Koha::AuthorisedValues object for location,ccode,lost and damaged.

For performance on huge number of checkouts :
Item types should be fetch once before the loop.
authorised values should call Koha::AuthorisedValues->get_description_by_koha_field because it uses a cache.

I've tested with Plack :
Without patch :
100 checkouts = 6 seconds
1000 checkouts = 60 seconds
With patch :
100 checkouts = 5 seconds
1000 checkouts = 44 seconds

Patch also changes the fact that authorised value categories are no longer hardcoded LOC,CCODE,LOST and DAMAGED, they depend on default framework.
Like is doing Bug 26323.

Test plan :
1) Dont apply patch
2) Use sql to define some items lost and damaged
3) Look at checkouts table on a patron with a lot of checkouts
4) Apply patch
5) Look at checkouts table again
6) Check infos are the same : record level item type, item type, location, collection, lost, damaged
7) Check infos are the same in "Number of checkouts by item type"

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 13 Katrin Fischer 2020-09-20 17:35:00 UTC
Created attachment 110429 [details] [review]
Bug 26424: Better performance of svc/checkouts

Ajax script svc/checkouts display checkouts of a patron.
For each item, it fetches a Koha::ItemType object and a Koha::AuthorisedValues object for location,ccode,lost and damaged.

For performance on huge number of checkouts :
Item types should be fetch once before the loop.
authorised values should call Koha::AuthorisedValues->get_description_by_koha_field because it uses a cache.

I've tested with Plack :
Without patch :
100 checkouts = 6 seconds
1000 checkouts = 60 seconds
With patch :
100 checkouts = 5 seconds
1000 checkouts = 44 seconds

Patch also changes the fact that authorised value categories are no longer hardcoded LOC,CCODE,LOST and DAMAGED, they depend on default framework.
Like is doing Bug 26323.

Test plan :
1) Dont apply patch
2) Use sql to define some items lost and damaged
3) Look at checkouts table on a patron with a lot of checkouts
4) Apply patch
5) Look at checkouts table again
6) Check infos are the same : record level item type, item type, location, collection, lost, damaged
7) Check infos are the same in "Number of checkouts by item type"

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 14 Jonathan Druart 2020-09-28 14:12:12 UTC
Pushed to master for 20.11, thanks to everybody involved!
Comment 15 Lucas Gass 2020-10-20 13:47:37 UTC
backported to 20.05.x for 20.05.05
Comment 16 Aleisha Amohia 2020-10-27 03:53:13 UTC
enhancement, not backported to 19.11.x