Bug 31846

Summary: Allow setting serials search results limit
Product: Koha Reporter: David Gustafsson <glasklas>
Component: SerialsAssignee: David Gustafsson <glasklas>
Status: RESOLVED FIXED QA Contact: Kyle M Hall (khall) <kyle>
Severity: enhancement    
Priority: P5 - low CC: caroline.cyr-la-rose, david, fridolin.somers, kyle, paul.poulain, tomascohen
Version: Main   
Hardware: All   
OS: All   
Change sponsored?: --- Patch complexity: ---
Documentation contact: Caroline Cyr La Rose Documentation submission: https://gitlab.com/koha-community/koha-manual/-/merge_requests/793
Text to go in the release notes:
Version(s) released in:
23.11.00
Circulation function:
Attachments: Bug 31846: Add syspref for serials search results limit
Bug 31846: Add syspref for serials search results limit
Bug 31846: Add syspref for serials search results limit
Bug 31846: Add syspref for serials search results limit
Bug 31846: (QA follow-up) Tidy code to make qa script happy

Description David Gustafsson 2022-10-17 15:49:32 UTC
Performing and advanced search on serials with a significant amount of subscriptions in the database can take up a lot of server resources and even (in our case) cause serious server overload. By introducing a syspref that allows setting a result limit this problem can be mitigated. Also discovered and fixed a lot of performance bottlenecks but will post this as a separate patch.
Comment 1 David Gustafsson 2022-10-17 16:54:29 UTC Comment hidden (obsolete)
Comment 2 David Gustafsson 2022-10-17 16:56:49 UTC
Created attachment 141999 [details] [review]
Bug 31846: Add syspref for serials search results limit

To test:

1) Make sure SerialsSearchResultsLimit syspref is unset or set to 0.
2) Perform an advanced search on serials without any conditions
   and confirm all serials are listed as expected.
3) Set SerialsSearchResultsLimit to a value less the the number
   of total subscriptions, perform the search again, and confirm
   that the number of serials has been limited to the set value.
4) Ensure all tests pass in t/db_dependent/Serials.t

Sponsored-by: Gothenburg University Library
Comment 3 Katrin Fischer 2022-10-17 19:44:22 UTC
That is something we don't do in Koha so far afaik. I wonder if moving the search to the API as it was done for the order receive page and the patron search would not be a better way forward here to fix this issue?
Comment 4 David Gustafsson 2022-10-18 13:30:21 UTC
(In reply to Katrin Fischer from comment #3)
> That is something we don't do in Koha so far afaik. I wonder if moving the
> search to the API as it was done for the order receive page and the patron
> search would not be a better way forward here to fix this issue?

Yes, but that would require a lot more work, especially since SearchSubscriptions relies on custom queries and in practice does use the Koha::Subscriptions DBIx interface. If someone would like to refactor the search page that would perhaps be preferable, but this is quite a simple fix and think could still be useful until someone puts in the effort. The current fix works for our current needs, without it a staff interface user could presently accidentally put such a high load on the server to cause a temporary system outage.
Comment 5 David Gustafsson 2022-10-18 14:00:50 UTC
Now created a separate issue for the performance improvements in bug 31856.
Comment 6 David Nind 2022-10-18 19:59:25 UTC
I've added David Gustafsson as the assignee.
Comment 7 ByWater Sandboxes 2023-08-17 12:38:45 UTC
Created attachment 154563 [details] [review]
Bug 31846: Add syspref for serials search results limit

To test:

1) Make sure SerialsSearchResultsLimit syspref is unset or set to 0.
2) Perform an advanced search on serials without any conditions
   and confirm all serials are listed as expected.
3) Set SerialsSearchResultsLimit to a value less the the number
   of total subscriptions, perform the search again, and confirm
   that the number of serials has been limited to the set value.
4) Ensure all tests pass in t/db_dependent/Serials.t

Sponsored-by: Gothenburg University Library
Signed-off-by: Kelly McElligott <kelly@bywatersolutions.com>
Comment 8 David Nind 2023-08-17 17:01:49 UTC
Hi Kelly.

Since you've signed it off, you should change the status in Bugzilla to 'Signed Off'.

That way you will get the credit in the dashboard!

David
Comment 9 Kyle M Hall (khall) 2023-10-13 13:56:48 UTC
Created attachment 157102 [details] [review]
Bug 31846: Add syspref for serials search results limit

To test:

1) Make sure SerialsSearchResultsLimit syspref is unset or set to 0.
2) Perform an advanced search on serials without any conditions
   and confirm all serials are listed as expected.
3) Set SerialsSearchResultsLimit to a value less the the number
   of total subscriptions, perform the search again, and confirm
   that the number of serials has been limited to the set value.
4) Ensure all tests pass in t/db_dependent/Serials.t

Sponsored-by: Gothenburg University Library
Signed-off-by: Kelly McElligott <kelly@bywatersolutions.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 10 Kyle M Hall (khall) 2023-10-13 13:56:55 UTC
Created attachment 157103 [details] [review]
Bug 31846: (QA follow-up) Tidy code to make qa script happy

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 11 Katrin Fischer 2023-10-13 17:14:09 UTC
(In reply to David Gustafsson from comment #4)
> (In reply to Katrin Fischer from comment #3)
> > That is something we don't do in Koha so far afaik. I wonder if moving the
> > search to the API as it was done for the order receive page and the patron
> > search would not be a better way forward here to fix this issue?
> 
> Yes, but that would require a lot more work, especially since
> SearchSubscriptions relies on custom queries and in practice does use the
> Koha::Subscriptions DBIx interface. If someone would like to refactor the
> search page that would perhaps be preferable, but this is quite a simple fix
> and think could still be useful until someone puts in the effort. The
> current fix works for our current needs, without it a staff interface user
> could presently accidentally put such a high load on the server to cause a
> temporary system outage.

There was some controversy here if we should implement it or not - Kyle, could you give your opinion?
Comment 12 Tomás Cohen Arazi (tcohen) 2023-10-25 13:36:16 UTC
Pushed to master for 23.11.

Nice work everyone, thanks!
Comment 13 Fridolin Somers 2023-10-26 07:56:53 UTC
Enhancement not pushed to 23.05.x