Bug 23391 - Hide finished ILL requests
Summary: Hide finished ILL requests
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: ILL (show other bugs)
Version: Main
Hardware: All All
: P5 - low enhancement
Assignee: Magnus Enger
QA Contact: Katrin Fischer
URL:
Keywords:
Depends on:
Blocks: 28641
  Show dependency tree / graph
 
Reported: 2019-07-29 10:09 UTC by Magnus Enger
Modified: 2023-06-08 22:26 UTC (History)
5 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Small patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
This adds a new system preference ILLHiddenRequestStatuses that takes a list of ILL status codes to be hidden from the ILL requests table in the ILL module. This allows to hide finished and cancelled ILL requests improving the performance of the table in busy libraries but also making it easier to keep track of the pending requests.
Version(s) released in:
20.11.00
Circulation function:

Attachments
Bug 23391 - Hide finished ILL requests (8.08 KB, patch)
2019-12-12 14:18 UTC, Magnus Enger 		Details | Diff | Splinter Review
Bug 23391 - Hide finished ILL requests (7.92 KB, patch)
2020-02-04 13:58 UTC, Magnus Enger 		Details | Diff | Splinter Review
Bug 23391 - Hide finished ILL requests (8.00 KB, patch)
2020-02-04 14:12 UTC, Andrew Isherwood 		Details | Diff | Splinter Review
Bug 23391: Hide finished ILL requests (8.00 KB, patch)
2020-06-29 16:40 UTC, Katrin Fischer 		Details | Diff | Splinter Review
Bug 23391: Hide finished ILL requests (8.06 KB, patch)
2020-07-06 16:19 UTC, Katrin Fischer 		Details | Diff | Splinter Review
Bug 23391: Remove warnings (1.50 KB, patch)
2020-07-20 15:45 UTC, Jonathan Druart 		Details | Diff | Splinter Review
Show Obsolete (4) View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Magnus Enger 2019-07-29 10:09:22 UTC 
The default for the main ILL page is to show all ILL requests. For active libraries this will grow to a huge list quite fast. We should hide resuests that are "done", and only show them when they are requested specifically. 

(I was sure there was a bug for this already, but can't for the life of me find it.)
Comment 1 Magnus Enger 2019-12-06 12:53:40 UTC 
This is getting to be a real problem for some of our more ILL-active libraries. 

One solution could be to just have a syspref that takes a list of statuses that should be hidden by default, perhaps?
Comment 2 Magnus Enger 2019-12-12 14:18:59 UTC Comment hidden (obsolete)
Comment 3 Magnus Enger 2019-12-12 14:21:28 UTC 
One point that should be considered: I take the string from the syspref and try to clean it up a bit with this syspref:

$hidden_statuses_string =~ s/[^0-9A-Za-z_|]//g;

This means that status code will only be able to contain numbers, lower and upper ascii letters and underscore. This is perhaps too strict?
Comment 4 Andrew Isherwood 2020-02-04 13:37:14 UTC 
This is a great idea Magnus, I somehow missed this bug when it first appeared.

I'm not sure of the value of cleaning up the codes to be tested for. Currently we have no restriction of status code formats, so the clean up could actually cause matching results to not be found.

Happy to test and sign this off, it'll be a great addition, a few of our customers are wanting something like this! :)
Comment 5 Magnus Enger 2020-02-04 13:44:24 UTC 
(In reply to Andrew Isherwood from comment #4)
> This is a great idea Magnus, I somehow missed this bug when it first
> appeared.
> 
> I'm not sure of the value of cleaning up the codes to be tested for.
> Currently we have no restriction of status code formats, so the clean up
> could actually cause matching results to not be found.

So do you think I should get rid of the cleanup?

> Happy to test and sign this off, it'll be a great addition, a few of our
> customers are wanting something like this! :)

Yay! :-)
Comment 6 Andrew Isherwood 2020-02-04 13:48:19 UTC 
(In reply to Magnus Enger from comment #5)
> (In reply to Andrew Isherwood from comment #4)

> So do you think I should get rid of the cleanup?

I'm just not sure what it gives us other than potentially breaking the matching. I think we have to trust users to be able to enter the status codes without messing them up. Of course, I could be entirely missing your intention with the cleanup!
Comment 7 Magnus Enger 2020-02-04 13:53:21 UTC 
(In reply to Andrew Isherwood from comment #6)
> I'm just not sure what it gives us other than potentially breaking the
> matching. I think we have to trust users to be able to enter the status
> codes without messing them up. Of course, I could be entirely missing your
> intention with the cleanup!

I'm not sure myself, any longer... I'll do a revised patch.
Comment 8 Magnus Enger 2020-02-04 13:58:41 UTC Comment hidden (obsolete)
Comment 9 Magnus Enger 2020-02-04 13:59:36 UTC 
New version of the patch is without the cleanup.
Comment 10 Andrew Isherwood 2020-02-04 14:12:10 UTC 
Created attachment 98391 [details] [review]
Bug 23391 - Hide finished ILL requests

Currently, the main table of ILL requests will display all ILL
requests in the database, regardless of their status. For libraries
with active ILL this quickly leads to a lot of requests being displayed,
and the main page of the ILL module taking a long time to load. This
patch proposes to fix this by introducing the ILLHiddenRequestStatuses
syspref, which can take a pipe-separated list of ILL statuses that
will be hidden from view in the ILL module. This means that the
only way to find a hidden request will be through a report.

To test:
- Apply the patch and make sure the atomic database update is run
- Make sure you have a few ILL requests, with at least two different
  statuses
- Check that all requests are still displayed in the main table of
  ILL requests
- Add one of the statuses* you have in your database to the
  ILLHiddenRequestStatuses syspref, reload the ILL module frontpage
  and verify that requests with the given status are not displayed
- Change the syspref to another status and verify requests with
  that status are now hidden
- Change the syspref to hold both statuses, separated by the pipe
  symbol (e.g.: A|B). Verify that no requests with the given
  statuses are now displayed
- Run the ILL REST API tests, e.g.:
  $ sudo koha-shell -c "prove t/db_dependent/api/v1/illrequests.t" kohadev

* = The ILLHiddenRequestStatuses syspref should hold status codes, like
"REQ" and "NEW", not their human readable counterparts.

Signed-off-by: Andrew Isherwood <andrew.isherwood@ptfs-europe.com>
Comment 11 Katrin Fischer 2020-06-29 16:40:34 UTC 
Created attachment 106411 [details] [review]
Bug 23391: Hide finished ILL requests

Currently, the main table of ILL requests will display all ILL
requests in the database, regardless of their status. For libraries
with active ILL this quickly leads to a lot of requests being displayed,
and the main page of the ILL module taking a long time to load. This
patch proposes to fix this by introducing the ILLHiddenRequestStatuses
syspref, which can take a pipe-separated list of ILL statuses that
will be hidden from view in the ILL module. This means that the
only way to find a hidden request will be through a report.

To test:
- Apply the patch and make sure the atomic database update is run
- Make sure you have a few ILL requests, with at least two different
  statuses
- Check that all requests are still displayed in the main table of
  ILL requests
- Add one of the statuses* you have in your database to the
  ILLHiddenRequestStatuses syspref, reload the ILL module frontpage
  and verify that requests with the given status are not displayed
- Change the syspref to another status and verify requests with
  that status are now hidden
- Change the syspref to hold both statuses, separated by the pipe
  symbol (e.g.: A|B). Verify that no requests with the given
  statuses are now displayed
- Run the ILL REST API tests, e.g.:
  $ sudo koha-shell -c "prove t/db_dependent/api/v1/illrequests.t" kohadev

* = The ILLHiddenRequestStatuses syspref should hold status codes, like
"REQ" and "NEW", not their human readable counterparts.

Signed-off-by: Andrew Isherwood <andrew.isherwood@ptfs-europe.com>
Comment 12 Katrin Fischer 2020-06-29 16:40:53 UTC 
Rebased and fixed commit message - will continue testing.
Comment 13 Katrin Fischer 2020-07-06 16:19:38 UTC 
Created attachment 106599 [details] [review]
Bug 23391: Hide finished ILL requests

Currently, the main table of ILL requests will display all ILL
requests in the database, regardless of their status. For libraries
with active ILL this quickly leads to a lot of requests being displayed,
and the main page of the ILL module taking a long time to load. This
patch proposes to fix this by introducing the ILLHiddenRequestStatuses
syspref, which can take a pipe-separated list of ILL statuses that
will be hidden from view in the ILL module. This means that the
only way to find a hidden request will be through a report.

To test:
- Apply the patch and make sure the atomic database update is run
- Make sure you have a few ILL requests, with at least two different
  statuses
- Check that all requests are still displayed in the main table of
  ILL requests
- Add one of the statuses* you have in your database to the
  ILLHiddenRequestStatuses syspref, reload the ILL module frontpage
  and verify that requests with the given status are not displayed
- Change the syspref to another status and verify requests with
  that status are now hidden
- Change the syspref to hold both statuses, separated by the pipe
  symbol (e.g.: A|B). Verify that no requests with the given
  statuses are now displayed
- Run the ILL REST API tests, e.g.:
  $ sudo koha-shell -c "prove t/db_dependent/api/v1/illrequests.t" kohadev

* = The ILLHiddenRequestStatuses syspref should hold status codes, like
"REQ" and "NEW", not their human readable counterparts.

Signed-off-by: Andrew Isherwood <andrew.isherwood@ptfs-europe.com>

Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Comment 14 Katrin Fischer 2020-07-06 16:21:03 UTC 
I've heard about huge performance issues here as well for busy libraries and I think this is a good first step to fix the issue.

In the future we could think about another option to pre-filter, but still allow to search with a limit on one of these status.
Comment 15 Jonathan Druart 2020-07-20 14:13:36 UTC 
The correct implementation would have been to move the statuses to a dedicated table. Then a "done" column could have replaced this syspref.

Also I think we should have a button to display the "hidden" requests.
Comment 16 Jonathan Druart 2020-07-20 15:45:07 UTC 
Created attachment 107121 [details] [review]
Bug 23391: Remove warnings

t/db_dependent/api/v1/illrequests.t .. Use of uninitialized value $hidden_statuses_string in split at /kohadevbox/koha/Koha/REST/V1/Illrequests.pm line 61.
Use of uninitialized value $hidden_statuses_string in split at /kohadevbox/koha/Koha/REST/V1/Illrequests.pm line 61.
Use of uninitialized value $hidden_statuses_string in split at /kohadevbox/koha/Koha/REST/V1/Illrequests.pm line 61.
Use of uninitialized value $hidden_statuses_string in split at /kohadevbox/koha/Koha/REST/V1/Illrequests.pm line 61.
Use of uninitialized value $hidden_statuses_string in split at /kohadevbox/koha/Koha/REST/V1/Illrequests.pm line 61.
t/db_dependent/api/v1/illrequests.t .. ok
Comment 17 Jonathan Druart 2020-07-20 15:48:08 UTC 
Pushed to master for 20.11, thanks to everybody involved!
Comment 18 Katrin Fischer 2020-07-20 20:47:07 UTC 
(In reply to Jonathan Druart from comment #15)
> The correct implementation would have been to move the statuses to a
> dedicated table. Then a "done" column could have replaced this syspref.
> 
> Also I think we should have a button to display the "hidden" requests.

I've filed bug 26030 for this.
Comment 19 Lucas Gass (lukeg) 2020-07-27 16:22:12 UTC 
I attempted to backport this to 20.05.x but I see that I am failing tests:

prove -v t/db_dependent/api/v1/illrequests.t


    #   Failed test '200 OK'
    #   at t/db_dependent/api/v1/illrequests.t line 179.
    #          got: '503'
    #     expected: '200'
    not ok 30 - exact match for JSON Pointer ""

    #   Failed test 'exact match for JSON Pointer ""'
    #   at t/db_dependent/api/v1/illrequests.t line 179.
    #     Structures begin differing at:
    #          $got = HASH(0x5621283b0dd0)
    #     $expected = ARRAY(0x5621283a8a60)
    # Looks like you failed 21 tests of 30.
not ok 1 - list() tests

#   Failed test 'list() tests'
#   at t/db_dependent/api/v1/illrequests.t line 183.
# Looks like you failed 1 test of 1.
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/1 subtests



I'd love to backport this if we could make the tests pass on 20.05. Until then, no backport