Bug 33430 - Use REST API for suggestions tables
Summary: Use REST API for suggestions tables
Status: Passed QA
Alias: None
Product: Koha
Classification: Unclassified
Component: Acquisitions (show other bugs)
Version: Main
Hardware: All All
: P5 - low major
Assignee: Matt Blenkinsop
QA Contact: Lisette Scheer
URL:
Keywords:
: 27183 (view as bug list)
Depends on:
Blocks: 34832
  Show dependency treegraph
 
Reported: 2023-04-06 11:30 UTC by Laura Escamilla
Modified: 2025-05-19 08:44 UTC (History)
21 users (show)

See Also:
GIT URL:
Change sponsored?: Sponsored
Patch complexity: Medium patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
This development reworks the purchase suggestion tables to use the API, which results in much faster load times when there are many suggestions.
Version(s) released in:
Circulation function:


Attachments
Bug 33430: Add strings_map method (5.32 KB, patch)
2025-05-09 07:19 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 33430: Update API specs to include required embeds (1.82 KB, patch)
2025-05-09 07:19 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 33430: Pass search parameters to the template (7.36 KB, patch)
2025-05-09 07:19 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 33430: Enable datatables for suggestion tabs (34.22 KB, patch)
2025-05-09 07:19 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 33430: DO NOT PUSH - data script (2.14 KB, patch)
2025-05-09 07:22 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 33430: Enable datatables for suggestion tabs (34.24 KB, patch)
2025-05-09 09:08 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 33430: Add strings_map method (5.38 KB, patch)
2025-05-09 16:43 UTC, Emily Lamancusa (emlam)
Details | Diff | Splinter Review
Bug 33430: Update API specs to include required embeds (1.89 KB, patch)
2025-05-09 16:44 UTC, Emily Lamancusa (emlam)
Details | Diff | Splinter Review
Bug 33430: Pass search parameters to the template (7.43 KB, patch)
2025-05-09 16:44 UTC, Emily Lamancusa (emlam)
Details | Diff | Splinter Review
Bug 33430: DO NOT PUSH - data script (2.21 KB, patch)
2025-05-09 16:44 UTC, Emily Lamancusa (emlam)
Details | Diff | Splinter Review
Bug 33430: Enable datatables for suggestion tabs (34.31 KB, patch)
2025-05-09 16:44 UTC, Emily Lamancusa (emlam)
Details | Diff | Splinter Review
Bug 33430: DO NOT PUSH - data script (2.27 KB, patch)
2025-05-09 17:18 UTC, Lisette Scheer
Details | Diff | Splinter Review
Bug 33430: Add strings_map method (5.45 KB, patch)
2025-05-09 17:18 UTC, Lisette Scheer
Details | Diff | Splinter Review
Bug 33430: Update API specs to include required embeds (1.95 KB, patch)
2025-05-09 17:18 UTC, Lisette Scheer
Details | Diff | Splinter Review
Bug 33430: Pass search parameters to the template (7.49 KB, patch)
2025-05-09 17:18 UTC, Lisette Scheer
Details | Diff | Splinter Review
Bug 33430: Enable datatables for suggestion tabs (34.37 KB, patch)
2025-05-09 17:18 UTC, Lisette Scheer
Details | Diff | Splinter Review
Bug 33430: DO NOT PUSH - data script (2.27 KB, patch)
2025-05-09 17:23 UTC, Lisette Scheer
Details | Diff | Splinter Review
Bug 33430: Add strings_map method (5.45 KB, patch)
2025-05-09 17:23 UTC, Lisette Scheer
Details | Diff | Splinter Review
Bug 33430: Update API specs to include required embeds (1.95 KB, patch)
2025-05-09 17:24 UTC, Lisette Scheer
Details | Diff | Splinter Review
Bug 33430: Pass search parameters to the template (7.49 KB, patch)
2025-05-09 17:24 UTC, Lisette Scheer
Details | Diff | Splinter Review
Bug 33430: Enable datatables for suggestion tabs (34.42 KB, patch)
2025-05-09 17:25 UTC, Lisette Scheer
Details | Diff | Splinter Review
Bug 33430: DO NOT PUSH - data script (2.32 KB, patch)
2025-05-13 09:39 UTC, Philip Orr
Details | Diff | Splinter Review
Bug 33430: Add strings_map method (5.49 KB, patch)
2025-05-13 09:39 UTC, Philip Orr
Details | Diff | Splinter Review
Bug 33430: Update API specs to include required embeds (2.00 KB, patch)
2025-05-13 09:39 UTC, Philip Orr
Details | Diff | Splinter Review
Bug 33430: Pass search parameters to the template (7.54 KB, patch)
2025-05-13 09:40 UTC, Philip Orr
Details | Diff | Splinter Review
Bug 33430: Enable datatables for suggestion tabs (34.47 KB, patch)
2025-05-13 09:40 UTC, Philip Orr
Details | Diff | Splinter Review
Bug 33430: DO NOT PUSH - data script (2.54 KB, patch)
2025-05-14 10:00 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 33430: Add strings_map method (5.32 KB, patch)
2025-05-14 10:00 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 33430: Update API specs to include required embeds (1.82 KB, patch)
2025-05-14 10:00 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 33430: Pass search parameters to the template (7.36 KB, patch)
2025-05-14 10:01 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 33430: Enable datatables for suggestion tabs (35.07 KB, patch)
2025-05-14 10:01 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 33430: (follow-up) Restore sorting to table (7.70 KB, patch)
2025-05-14 10:01 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 33430: (follow-up) Fix the data filtering from the left menu (3.18 KB, patch)
2025-05-14 10:01 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 33430: (follow-up) Handle undefined data in column renders (4.48 KB, patch)
2025-05-15 08:56 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 33430: DO NOT PUSH - data script (2.60 KB, patch)
2025-05-15 09:05 UTC, Fridolin Somers
Details | Diff | Splinter Review
Bug 33430: Add strings_map method (5.37 KB, patch)
2025-05-15 09:05 UTC, Fridolin Somers
Details | Diff | Splinter Review
Bug 33430: Update API specs to include required embeds (1.88 KB, patch)
2025-05-15 09:05 UTC, Fridolin Somers
Details | Diff | Splinter Review
Bug 33430: Pass search parameters to the template (7.42 KB, patch)
2025-05-15 09:05 UTC, Fridolin Somers
Details | Diff | Splinter Review
Bug 33430: Enable datatables for suggestion tabs (35.13 KB, patch)
2025-05-15 09:05 UTC, Fridolin Somers
Details | Diff | Splinter Review
Bug 33430: (follow-up) Restore sorting to table (7.75 KB, patch)
2025-05-15 09:06 UTC, Fridolin Somers
Details | Diff | Splinter Review
Bug 33430: (follow-up) Fix the data filtering from the left menu (3.24 KB, patch)
2025-05-15 09:06 UTC, Fridolin Somers
Details | Diff | Splinter Review
Bug 33430: (follow-up) Handle undefined data in column renders (4.54 KB, patch)
2025-05-15 09:06 UTC, Fridolin Somers
Details | Diff | Splinter Review
Bug 33430: DO NOT PUSH - data script (2.67 KB, patch)
2025-05-15 16:58 UTC, Lisette Scheer
Details | Diff | Splinter Review
Bug 33430: Add strings_map method (5.44 KB, patch)
2025-05-15 16:58 UTC, Lisette Scheer
Details | Diff | Splinter Review
Bug 33430: Update API specs to include required embeds (1.94 KB, patch)
2025-05-15 16:58 UTC, Lisette Scheer
Details | Diff | Splinter Review
Bug 33430: Pass search parameters to the template (7.48 KB, patch)
2025-05-15 16:58 UTC, Lisette Scheer
Details | Diff | Splinter Review
Bug 33430: Enable datatables for suggestion tabs (35.19 KB, patch)
2025-05-15 16:58 UTC, Lisette Scheer
Details | Diff | Splinter Review
Bug 33430: (follow-up) Restore sorting to table (7.82 KB, patch)
2025-05-15 16:58 UTC, Lisette Scheer
Details | Diff | Splinter Review
Bug 33430: (follow-up) Fix the data filtering from the left menu (3.30 KB, patch)
2025-05-15 16:58 UTC, Lisette Scheer
Details | Diff | Splinter Review
Bug 33430: (follow-up) Handle undefined data in column renders (4.60 KB, patch)
2025-05-15 16:58 UTC, Lisette Scheer
Details | Diff | Splinter Review
Bug 33430: (follow-up) Fix API spec terminology (966 bytes, patch)
2025-05-19 08:43 UTC, Matt Blenkinsop
Details | Diff | Splinter Review
Bug 33430: (follow-up) Prevent anonymous suggestions from breaking the table (3.50 KB, patch)
2025-05-19 08:43 UTC, Matt Blenkinsop
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Laura Escamilla 2023-04-06 11:30:24 UTC
The purchase suggestion management page take anywhere between 15-45 seconds to load if there are several hundred items in the suggestions table.
Comment 1 Lucas Gass (lukeg) 2023-04-06 18:42:00 UTC
For testing purposes in koha-testing-docker I wrote this. Definitely DO NOT add this to a production server:

$('body').append('<button id="makeSuggestions">Make 100 suggestions</button>');
  $('#makeSuggestions').click( function() {
    for (let i = 0; i < 100; i++) {
      let branches = ["CPL", "FFL","FPL","FRL","IPT","LPL","MPL","PVL","RPL","SPL","TPL","UPL"];
      let random_branch = Math.floor(Math.random() * branches.length);

      let statuses = ["ASKED", "ACCEPTED", "REJECTED"];
      let random_status = Math.floor(Math.random() * statuses.length);

      let random_title = (Math.random() + 1).toString(36).substring(2);
      $.ajax({
        url: '/api/v1/suggestions',
        method: 'POST',
        data: JSON.stringify({
              "title": random_title,
              "item_type": "BK",
              "suggested_by": Math.floor(Math.random() * 51),
              "suggestion_date": "2023-04-06",
              "timestamp": "2023-04-06T17:59:26+00:00",
              "managed_by": 51,
              "library_id": branches[random_branch],
              "status": statuses[random_status],
      }),
        contentType: 'application/json',
        success: function(result) {
            console.log('suggestion added');
        },
    });
  } 
});
Comment 2 Lucas Gass (lukeg) 2023-04-26 17:33:20 UTC
Ultimately these tables probably need to be coming for the REST API?
Comment 3 Felicity Brown 2024-01-05 00:05:04 UTC
Would it be possible to have batch changes made by queuing them, like we have for batch item modification? So we could continue to work on other purchase suggestions while the last set of edits are updating? I find it grinds particularly badly when I'm trying to update several records from a page to assign a manager to them.
Comment 4 Candice Hope 2024-01-24 18:56:20 UTC
This is very slow on our system too. Hopefully there is a good solution for it soon.
Comment 5 Joe Askins 2024-06-11 20:07:45 UTC
Fayetteville Public Library cosigning the need for this fix -- the slow load times are making it very difficult to work through these purchase suggestions.
Comment 6 Teresa Dahlgren 2024-07-24 19:47:00 UTC
This is a challenge at our library as well. The purchase suggestion management page takes a long time to load, and it also takes a long time to save changes made to a purchase suggestion (such as changing the status to "Accepted" or "Ordered").
Comment 7 CJ Lynce 2024-07-26 14:21:12 UTC
We are experiencing this as well, with just over 400 items between the Accepted, Checked, and Rejected tabs. The pages can take 10+ seconds, and at times even longer to load.

We also suspect this is related to an issue that, after modifying a suggestion, the page returns to the suggestions list, but never fully loads the table of suggestions (We cannot conclusively prove this is related at this point, but are trying to narrow this down)
Comment 8 Lucas Gass (lukeg) 2024-07-26 14:27:25 UTC
*** Bug 27183 has been marked as a duplicate of this bug. ***
Comment 9 Lucas Gass (lukeg) 2024-07-26 14:29:37 UTC
Looks like Bug 27183 is a duplicate of this one. We load all the suggestions each time visiting the page. As Michal pointed out, API pagination can solve this. Not easy though. :)
Comment 10 Ray Delahunty 2024-08-13 12:25:25 UTC
Is Bug 31024 also a duplicate of this one? We have massively configured Suggestions Management with 5 additional SUGGEST_STATUS (and tweaks to the default ones). We have added various notices so we offer both centralised and college-based Suggestions Management using very different workflows. The feature is now widely used by many staff and it would be great to have the page draw quickly.
Comment 11 Katrin Fischer 2024-08-13 12:34:23 UTC
The move to REST is the ultimate fix, but could we improve performance otherwise too? How are we doing with indexes on the table and the like, are there some easy improvements we could make here?

If not, I think marking as duplicate would be the next logical step.
Comment 12 Christopher Brannon 2025-04-30 17:29:24 UTC
Oh, PLEASE, fix purchase suggestions!  I am losing too much of my life waiting for   these tables to load.  It is a painstaking process right now.
Comment 13 Candice Hope 2025-04-30 17:32:54 UTC
I second that Christopher!
Comment 14 Kris 2025-04-30 17:49:53 UTC
Our staff would really love for the Suggestions table to work faster as well, they feel your pain Christopher.
Comment 15 Teresa Dahlgren 2025-04-30 18:32:23 UTC
I third that! It takes FOREVER to load and edit these!
Comment 16 Matt Blenkinsop 2025-05-09 07:19:19 UTC Comment hidden (obsolete)
Comment 17 Matt Blenkinsop 2025-05-09 07:19:23 UTC Comment hidden (obsolete)
Comment 18 Matt Blenkinsop 2025-05-09 07:19:27 UTC Comment hidden (obsolete)
Comment 19 Matt Blenkinsop 2025-05-09 07:19:31 UTC Comment hidden (obsolete)
Comment 20 Matt Blenkinsop 2025-05-09 07:22:24 UTC Comment hidden (obsolete)
Comment 21 Matt Blenkinsop 2025-05-09 09:08:32 UTC Comment hidden (obsolete)
Comment 22 Emily Lamancusa (emlam) 2025-05-09 16:43:57 UTC Comment hidden (obsolete)
Comment 23 Emily Lamancusa (emlam) 2025-05-09 16:44:00 UTC Comment hidden (obsolete)
Comment 24 Emily Lamancusa (emlam) 2025-05-09 16:44:04 UTC Comment hidden (obsolete)
Comment 25 Emily Lamancusa (emlam) 2025-05-09 16:44:07 UTC Comment hidden (obsolete)
Comment 26 Emily Lamancusa (emlam) 2025-05-09 16:44:11 UTC Comment hidden (obsolete)
Comment 27 Emily Lamancusa (emlam) 2025-05-09 16:51:14 UTC
This is a huge improvement! Everything worked as expected for me, and it's nice that each individual tab both loads on-demand and persists without an extra reload when switching between tabs. Signing off :)
Comment 28 Lisette Scheer 2025-05-09 17:18:17 UTC Comment hidden (obsolete)
Comment 29 Lisette Scheer 2025-05-09 17:18:21 UTC Comment hidden (obsolete)
Comment 30 Lisette Scheer 2025-05-09 17:18:28 UTC Comment hidden (obsolete)
Comment 31 Lisette Scheer 2025-05-09 17:18:32 UTC Comment hidden (obsolete)
Comment 32 Lisette Scheer 2025-05-09 17:18:36 UTC Comment hidden (obsolete)
Comment 33 Lisette Scheer 2025-05-09 17:23:16 UTC
Created attachment 182207 [details] [review]
Bug 33430: DO NOT PUSH - data script

Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>

Sponsored by: Montgomery County Public Libraries
Comment 34 Lisette Scheer 2025-05-09 17:23:46 UTC Comment hidden (obsolete)
Comment 35 Lisette Scheer 2025-05-09 17:24:04 UTC Comment hidden (obsolete)
Comment 36 Lisette Scheer 2025-05-09 17:24:25 UTC Comment hidden (obsolete)
Comment 37 Lisette Scheer 2025-05-09 17:25:05 UTC Comment hidden (obsolete)
Comment 38 Lisette Scheer 2025-05-09 17:26:18 UTC
Worked great, the only thing I noticed was that if you changed the page number and moved to a different tab, the page number was still changed. I believe this is a data tables bug not specific to this enhancement.
Comment 39 Philip Orr 2025-05-13 09:39:46 UTC
Created attachment 182339 [details] [review]
Bug 33430: DO NOT PUSH - data script

Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: krimsonkharne <philip.orr@posteo.de>
Comment 40 Philip Orr 2025-05-13 09:39:52 UTC
Created attachment 182340 [details] [review]
Bug 33430: Add strings_map method

This patch adds a strings_map method to allow the API to return the required authorised values needed for table displays

Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: krimsonkharne <philip.orr@posteo.de>
Comment 41 Philip Orr 2025-05-13 09:39:59 UTC
Created attachment 182341 [details] [review]
Bug 33430: Update API specs to include required embeds

Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: krimsonkharne <philip.orr@posteo.de>
Comment 42 Philip Orr 2025-05-13 09:40:12 UTC
Created attachment 182342 [details] [review]
Bug 33430: Pass search parameters to the template

This patch removes the existing process intensive searches and moves the search logic to a single group query that returns the values that will be the counts in the tabs. It then formats the parameters for each type and passes them to the template for the datatables ajax requests

Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: krimsonkharne <philip.orr@posteo.de>
Comment 43 Philip Orr 2025-05-13 09:40:17 UTC
Created attachment 182343 [details] [review]
Bug 33430: Enable datatables for suggestion tabs

Test plan:
1) Either add loads of suggestions to your KTD or use the script in the data script patch uploaded to the bug
2) Navigate to Acquisitions > Suggestions
3) Depending on how many suggestions you have loaded (the script does 2000), the page will load very slowly
4) Apply patches
5) yarn build && restart_all
6) Hard refresh the page, the page should load much faster and the first tab should load a paginated datatable
7) Click into another tab, the tab should also load a paginated datatable
8) Click back into the first tab, the original table should persist and should not reload
9) Use the edit tools at the bottom of the page, they should work as expected
10) Click on the links in various table cells, they should redirect as expected
11) Use the actions buttons in the far right column, then should work as expected

Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>

Sponsored by: Montgomery County Public Libraries

Signed-off-by: krimsonkharne <philip.orr@posteo.de>
Comment 44 Jonathan Druart 2025-05-13 12:12:17 UTC
This is not working for me, I am getting 400

search, biblio, title="h"

{"error":1,"error_code":"invalid_query"}
Comment 45 Jonathan Druart 2025-05-13 12:16:22 UTC
This does not seem correct: https://snipboard.io/W0kX5S.jpg
Where are the other tabs?
Comment 46 Matt Blenkinsop 2025-05-14 10:00:48 UTC
Created attachment 182428 [details] [review]
Bug 33430: DO NOT PUSH - data script
Comment 47 Matt Blenkinsop 2025-05-14 10:00:53 UTC
Created attachment 182429 [details] [review]
Bug 33430: Add strings_map method

This patch adds a strings_map method to allow the API to return the required authorised values needed for table displays
Comment 48 Matt Blenkinsop 2025-05-14 10:00:58 UTC
Created attachment 182430 [details] [review]
Bug 33430: Update API specs to include required embeds
Comment 49 Matt Blenkinsop 2025-05-14 10:01:04 UTC
Created attachment 182431 [details] [review]
Bug 33430: Pass search parameters to the template

This patch removes the existing process intensive searches and moves the search logic to a single group query that returns the values that will be the counts in the tabs. It then formats the parameters for each type and passes them to the template for the datatables ajax requests
Comment 50 Matt Blenkinsop 2025-05-14 10:01:09 UTC
Created attachment 182432 [details] [review]
Bug 33430: Enable datatables for suggestion tabs

Test plan:
1) Either add loads of suggestions to your KTD or use the script in the data script patch uploaded to the bug
2) Navigate to Acquisitions > Suggestions
3) Depending on how many suggestions you have loaded (the script does 2000), the page will load very slowly
4) Apply patches
5) yarn build && restart_all
6) Hard refresh the page, the page should load much faster and the first tab should load a paginated datatable
7) Click into another tab, the tab should also load a paginated datatable
8) Click back into the first tab, the original table should persist and should not reload
9) Use the edit tools at the bottom of the page, they should work as expected
10) Click on the links in various table cells, they should redirect as expected
11) Use the actions buttons in the far right column, then should work as expected
Comment 51 Matt Blenkinsop 2025-05-14 10:01:15 UTC
Created attachment 182433 [details] [review]
Bug 33430: (follow-up) Restore sorting to table
Comment 52 Matt Blenkinsop 2025-05-14 10:01:20 UTC
Created attachment 182434 [details] [review]
Bug 33430: (follow-up) Fix the data filtering from the left menu
Comment 53 Fridolin Somers 2025-05-15 08:25:46 UTC
I'm testing.

Looks great.

I just found one error.

Create a suggestion with library = "Any".
Display suggestions with "Viewing suggestions for library"
= "Any"
Table crashes with :
Uncaught TypeError: row.library is null
Comment 54 Fridolin Somers 2025-05-15 08:31:58 UTC
Also :
Create a suggestion from OPAC with only title
Look at suggestions table in staff interface
It crashes with :
Uncaught TypeError: row.manager is null
Comment 55 Matt Blenkinsop 2025-05-15 08:56:48 UTC
Created attachment 182478 [details] [review]
Bug 33430: (follow-up) Handle undefined data in column renders
Comment 56 Fridolin Somers 2025-05-15 09:05:38 UTC
Created attachment 182480 [details] [review]
Bug 33430: DO NOT PUSH - data script

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Comment 57 Fridolin Somers 2025-05-15 09:05:43 UTC
Created attachment 182481 [details] [review]
Bug 33430: Add strings_map method

This patch adds a strings_map method to allow the API to return the required authorised values needed for table displays

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Comment 58 Fridolin Somers 2025-05-15 09:05:48 UTC
Created attachment 182482 [details] [review]
Bug 33430: Update API specs to include required embeds

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Comment 59 Fridolin Somers 2025-05-15 09:05:52 UTC
Created attachment 182483 [details] [review]
Bug 33430: Pass search parameters to the template

This patch removes the existing process intensive searches and moves the search logic to a single group query that returns the values that will be the counts in the tabs. It then formats the parameters for each type and passes them to the template for the datatables ajax requests

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Comment 60 Fridolin Somers 2025-05-15 09:05:58 UTC
Created attachment 182484 [details] [review]
Bug 33430: Enable datatables for suggestion tabs

Test plan:
1) Either add loads of suggestions to your KTD or use the script in the data script patch uploaded to the bug
2) Navigate to Acquisitions > Suggestions
3) Depending on how many suggestions you have loaded (the script does 2000), the page will load very slowly
4) Apply patches
5) yarn build && restart_all
6) Hard refresh the page, the page should load much faster and the first tab should load a paginated datatable
7) Click into another tab, the tab should also load a paginated datatable
8) Click back into the first tab, the original table should persist and should not reload
9) Use the edit tools at the bottom of the page, they should work as expected
10) Click on the links in various table cells, they should redirect as expected
11) Use the actions buttons in the far right column, then should work as expected

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Comment 61 Fridolin Somers 2025-05-15 09:06:04 UTC
Created attachment 182485 [details] [review]
Bug 33430: (follow-up) Restore sorting to table

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Comment 62 Fridolin Somers 2025-05-15 09:06:11 UTC
Created attachment 182486 [details] [review]
Bug 33430: (follow-up) Fix the data filtering from the left menu

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Comment 63 Fridolin Somers 2025-05-15 09:06:15 UTC
Created attachment 182487 [details] [review]
Bug 33430: (follow-up) Handle undefined data in column renders

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Comment 64 Lisette Scheer 2025-05-15 16:58:22 UTC
Created attachment 182493 [details] [review]
Bug 33430: DO NOT PUSH - data script

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Comment 65 Lisette Scheer 2025-05-15 16:58:25 UTC
Created attachment 182494 [details] [review]
Bug 33430: Add strings_map method

This patch adds a strings_map method to allow the API to return the required authorised values needed for table displays

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Comment 66 Lisette Scheer 2025-05-15 16:58:29 UTC
Created attachment 182495 [details] [review]
Bug 33430: Update API specs to include required embeds

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Comment 67 Lisette Scheer 2025-05-15 16:58:33 UTC
Created attachment 182496 [details] [review]
Bug 33430: Pass search parameters to the template

This patch removes the existing process intensive searches and moves the search logic to a single group query that returns the values that will be the counts in the tabs. It then formats the parameters for each type and passes them to the template for the datatables ajax requests

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Comment 68 Lisette Scheer 2025-05-15 16:58:37 UTC
Created attachment 182497 [details] [review]
Bug 33430: Enable datatables for suggestion tabs

Test plan:
1) Either add loads of suggestions to your KTD or use the script in the data script patch uploaded to the bug
2) Navigate to Acquisitions > Suggestions
3) Depending on how many suggestions you have loaded (the script does 2000), the page will load very slowly
4) Apply patches
5) yarn build && restart_all
6) Hard refresh the page, the page should load much faster and the first tab should load a paginated datatable
7) Click into another tab, the tab should also load a paginated datatable
8) Click back into the first tab, the original table should persist and should not reload
9) Use the edit tools at the bottom of the page, they should work as expected
10) Click on the links in various table cells, they should redirect as expected
11) Use the actions buttons in the far right column, then should work as expected

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Comment 69 Lisette Scheer 2025-05-15 16:58:41 UTC
Created attachment 182498 [details] [review]
Bug 33430: (follow-up) Restore sorting to table

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Comment 70 Lisette Scheer 2025-05-15 16:58:44 UTC
Created attachment 182499 [details] [review]
Bug 33430: (follow-up) Fix the data filtering from the left menu

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Comment 71 Lisette Scheer 2025-05-15 16:58:48 UTC
Created attachment 182500 [details] [review]
Bug 33430: (follow-up) Handle undefined data in column renders

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Comment 72 Katrin Fischer 2025-05-16 14:38:59 UTC
1) Terminology

+    description: last member to modify the suggestion
+  library:
+    type:
+      - object
+      - "null"
+    description: branch the suggestion belongs to

last user/patron to...
library the suggestion...

I wanted to fix these in a follow-up, but then found a more severe issue:


2) Anonymous suggestions break things

To test:
AnonymousPatron = 2
AnonSuggestions = Allow

Add a suggestion from the OPAC without being logged in.
The table of suggestions won't load:

Organize by: Status
Tab: Pending

Uncaught TypeError: row.suggester is null
Comment 73 Katrin Fischer 2025-05-16 14:45:41 UTC
If this is an easy fix I'm inclined to try again Monday, but we need to make the decision that day.
Comment 74 Katrin Fischer 2025-05-16 14:51:07 UTC
3) Do dates sort correctly?

I couldn't fully verify and I am not sure how the mark-up should like for datatables driven by API.

I notice that before we had something like:

<td data-order="2023-03-15" class="dt-type-date">
 15.03.2023
 </td>

Now we have just:
<td>05.01.2025</td>
Comment 75 Matt Blenkinsop 2025-05-19 08:43:02 UTC
Created attachment 182577 [details] [review]
Bug 33430: (follow-up) Fix API spec terminology
Comment 76 Matt Blenkinsop 2025-05-19 08:43:07 UTC
Created attachment 182578 [details] [review]
Bug 33430: (follow-up) Prevent anonymous suggestions from breaking the table
Comment 77 Matt Blenkinsop 2025-05-19 08:44:20 UTC
I've fixed points 1 and 2, as far as I can tell the dates are sorting correctly for point 3 as well. I created a range of suggestions with different dates and using the column sorting was correctly ordering the dates