The purchase suggestion management page take anywhere between 15-45 seconds to load if there are several hundred items in the suggestions table.
For testing purposes in koha-testing-docker I wrote this. Definitely DO NOT add this to a production server: $('body').append('<button id="makeSuggestions">Make 100 suggestions</button>'); $('#makeSuggestions').click( function() { for (let i = 0; i < 100; i++) { let branches = ["CPL", "FFL","FPL","FRL","IPT","LPL","MPL","PVL","RPL","SPL","TPL","UPL"]; let random_branch = Math.floor(Math.random() * branches.length); let statuses = ["ASKED", "ACCEPTED", "REJECTED"]; let random_status = Math.floor(Math.random() * statuses.length); let random_title = (Math.random() + 1).toString(36).substring(2); $.ajax({ url: '/api/v1/suggestions', method: 'POST', data: JSON.stringify({ "title": random_title, "item_type": "BK", "suggested_by": Math.floor(Math.random() * 51), "suggestion_date": "2023-04-06", "timestamp": "2023-04-06T17:59:26+00:00", "managed_by": 51, "library_id": branches[random_branch], "status": statuses[random_status], }), contentType: 'application/json', success: function(result) { console.log('suggestion added'); }, }); } });
Ultimately these tables probably need to be coming for the REST API?
Would it be possible to have batch changes made by queuing them, like we have for batch item modification? So we could continue to work on other purchase suggestions while the last set of edits are updating? I find it grinds particularly badly when I'm trying to update several records from a page to assign a manager to them.
This is very slow on our system too. Hopefully there is a good solution for it soon.
Fayetteville Public Library cosigning the need for this fix -- the slow load times are making it very difficult to work through these purchase suggestions.
This is a challenge at our library as well. The purchase suggestion management page takes a long time to load, and it also takes a long time to save changes made to a purchase suggestion (such as changing the status to "Accepted" or "Ordered").
We are experiencing this as well, with just over 400 items between the Accepted, Checked, and Rejected tabs. The pages can take 10+ seconds, and at times even longer to load. We also suspect this is related to an issue that, after modifying a suggestion, the page returns to the suggestions list, but never fully loads the table of suggestions (We cannot conclusively prove this is related at this point, but are trying to narrow this down)
*** Bug 27183 has been marked as a duplicate of this bug. ***
Looks like Bug 27183 is a duplicate of this one. We load all the suggestions each time visiting the page. As Michal pointed out, API pagination can solve this. Not easy though. :)
Is Bug 31024 also a duplicate of this one? We have massively configured Suggestions Management with 5 additional SUGGEST_STATUS (and tweaks to the default ones). We have added various notices so we offer both centralised and college-based Suggestions Management using very different workflows. The feature is now widely used by many staff and it would be great to have the page draw quickly.
The move to REST is the ultimate fix, but could we improve performance otherwise too? How are we doing with indexes on the table and the like, are there some easy improvements we could make here? If not, I think marking as duplicate would be the next logical step.
Oh, PLEASE, fix purchase suggestions! I am losing too much of my life waiting for these tables to load. It is a painstaking process right now.
I second that Christopher!
Our staff would really love for the Suggestions table to work faster as well, they feel your pain Christopher.
I third that! It takes FOREVER to load and edit these!
Created attachment 182146 [details] [review] Bug 33430: Add strings_map method This patch adds a strings_map method to allow the API to return the required authorised values needed for table displays
Created attachment 182147 [details] [review] Bug 33430: Update API specs to include required embeds
Created attachment 182148 [details] [review] Bug 33430: Pass search parameters to the template This patch removes the existing process intensive searches and moves the search logic to a single group query that returns the values that will be the counts in the tabs. It then formats the parameters for each type and passes them to the template for the datatables ajax requests
Created attachment 182149 [details] [review] Bug 33430: Enable datatables for suggestion tabs Test plan: 1) Either add loads of suggestions to your KTD or use the script in the data script patch uploaded to the bug 2) Navigate to Acquisitions > Suggestions 3) Depending on how many suggestions you have loaded (the script does 2000), the page will load very slowly 4) Apply patches 5) yarn build && restart_all 6) Hard refresh the page, the page should load much faster and the first tab should load a paginated datatable 7) Click into another tab, the tab should also load a paginated datatable 8) Click back into the first tab, the original table should persist and should not reload 9) Use the edit tools at the bottom of the page, they should work as expected 10) Click on the links in various table cells, they should redirect as expected 11) Use the actions buttons in the far right column, then should work as expected
Created attachment 182150 [details] [review] Bug 33430: DO NOT PUSH - data script
Created attachment 182161 [details] [review] Bug 33430: Enable datatables for suggestion tabs Test plan: 1) Either add loads of suggestions to your KTD or use the script in the data script patch uploaded to the bug 2) Navigate to Acquisitions > Suggestions 3) Depending on how many suggestions you have loaded (the script does 2000), the page will load very slowly 4) Apply patches 5) yarn build && restart_all 6) Hard refresh the page, the page should load much faster and the first tab should load a paginated datatable 7) Click into another tab, the tab should also load a paginated datatable 8) Click back into the first tab, the original table should persist and should not reload 9) Use the edit tools at the bottom of the page, they should work as expected 10) Click on the links in various table cells, they should redirect as expected 11) Use the actions buttons in the far right column, then should work as expected
Created attachment 182196 [details] [review] Bug 33430: Add strings_map method This patch adds a strings_map method to allow the API to return the required authorised values needed for table displays Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Created attachment 182197 [details] [review] Bug 33430: Update API specs to include required embeds Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Created attachment 182198 [details] [review] Bug 33430: Pass search parameters to the template This patch removes the existing process intensive searches and moves the search logic to a single group query that returns the values that will be the counts in the tabs. It then formats the parameters for each type and passes them to the template for the datatables ajax requests Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Created attachment 182199 [details] [review] Bug 33430: DO NOT PUSH - data script Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Created attachment 182200 [details] [review] Bug 33430: Enable datatables for suggestion tabs Test plan: 1) Either add loads of suggestions to your KTD or use the script in the data script patch uploaded to the bug 2) Navigate to Acquisitions > Suggestions 3) Depending on how many suggestions you have loaded (the script does 2000), the page will load very slowly 4) Apply patches 5) yarn build && restart_all 6) Hard refresh the page, the page should load much faster and the first tab should load a paginated datatable 7) Click into another tab, the tab should also load a paginated datatable 8) Click back into the first tab, the original table should persist and should not reload 9) Use the edit tools at the bottom of the page, they should work as expected 10) Click on the links in various table cells, they should redirect as expected 11) Use the actions buttons in the far right column, then should work as expected Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
This is a huge improvement! Everything worked as expected for me, and it's nice that each individual tab both loads on-demand and persists without an extra reload when switching between tabs. Signing off :)
Created attachment 182202 [details] [review] Bug 33430: DO NOT PUSH - data script Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov> Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com> Sponsored by: Montgomery County Public Libraries
Created attachment 182203 [details] [review] Bug 33430: Add strings_map method This patch adds a strings_map method to allow the API to return the required authorised values needed for table displays Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov> Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Created attachment 182204 [details] [review] Bug 33430: Update API specs to include required embeds Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov> Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Created attachment 182205 [details] [review] Bug 33430: Pass search parameters to the template This patch removes the existing process intensive searches and moves the search logic to a single group query that returns the values that will be the counts in the tabs. It then formats the parameters for each type and passes them to the template for the datatables ajax requests Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov> Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Created attachment 182206 [details] [review] Bug 33430: Enable datatables for suggestion tabs Test plan: 1) Either add loads of suggestions to your KTD or use the script in the data script patch uploaded to the bug 2) Navigate to Acquisitions > Suggestions 3) Depending on how many suggestions you have loaded (the script does 2000), the page will load very slowly 4) Apply patches 5) yarn build && restart_all 6) Hard refresh the page, the page should load much faster and the first tab should load a paginated datatable 7) Click into another tab, the tab should also load a paginated datatable 8) Click back into the first tab, the original table should persist and should not reload 9) Use the edit tools at the bottom of the page, they should work as expected 10) Click on the links in various table cells, they should redirect as expected 11) Use the actions buttons in the far right column, then should work as expected Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov> Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Created attachment 182207 [details] [review] Bug 33430: DO NOT PUSH - data script Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov> Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com> Sponsored by: Montgomery County Public Libraries
Created attachment 182208 [details] [review] Bug 33430: Add strings_map method This patch adds a strings_map method to allow the API to return the required authorised values needed for table displays Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov> Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com> Sponsored by: Montgomery County Public Libraries
Created attachment 182209 [details] [review] Bug 33430: Update API specs to include required embeds Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov> Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com> Sponsored by: Montgomery County Public Libraries
Created attachment 182210 [details] [review] Bug 33430: Pass search parameters to the template This patch removes the existing process intensive searches and moves the search logic to a single group query that returns the values that will be the counts in the tabs. It then formats the parameters for each type and passes them to the template for the datatables ajax requests Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov> Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com> Sponsored by: Montgomery County Public Libraries
Created attachment 182211 [details] [review] Bug 33430: Enable datatables for suggestion tabs Test plan: 1) Either add loads of suggestions to your KTD or use the script in the data script patch uploaded to the bug 2) Navigate to Acquisitions > Suggestions 3) Depending on how many suggestions you have loaded (the script does 2000), the page will load very slowly 4) Apply patches 5) yarn build && restart_all 6) Hard refresh the page, the page should load much faster and the first tab should load a paginated datatable 7) Click into another tab, the tab should also load a paginated datatable 8) Click back into the first tab, the original table should persist and should not reload 9) Use the edit tools at the bottom of the page, they should work as expected 10) Click on the links in various table cells, they should redirect as expected 11) Use the actions buttons in the far right column, then should work as expected Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov> Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com> Sponsored by: Montgomery County Public Libraries
Worked great, the only thing I noticed was that if you changed the page number and moved to a different tab, the page number was still changed. I believe this is a data tables bug not specific to this enhancement.
Created attachment 182339 [details] [review] Bug 33430: DO NOT PUSH - data script Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov> Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com> Signed-off-by: krimsonkharne <philip.orr@posteo.de>
Created attachment 182340 [details] [review] Bug 33430: Add strings_map method This patch adds a strings_map method to allow the API to return the required authorised values needed for table displays Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov> Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com> Signed-off-by: krimsonkharne <philip.orr@posteo.de>
Created attachment 182341 [details] [review] Bug 33430: Update API specs to include required embeds Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov> Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com> Signed-off-by: krimsonkharne <philip.orr@posteo.de>
Created attachment 182342 [details] [review] Bug 33430: Pass search parameters to the template This patch removes the existing process intensive searches and moves the search logic to a single group query that returns the values that will be the counts in the tabs. It then formats the parameters for each type and passes them to the template for the datatables ajax requests Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov> Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com> Signed-off-by: krimsonkharne <philip.orr@posteo.de>
Created attachment 182343 [details] [review] Bug 33430: Enable datatables for suggestion tabs Test plan: 1) Either add loads of suggestions to your KTD or use the script in the data script patch uploaded to the bug 2) Navigate to Acquisitions > Suggestions 3) Depending on how many suggestions you have loaded (the script does 2000), the page will load very slowly 4) Apply patches 5) yarn build && restart_all 6) Hard refresh the page, the page should load much faster and the first tab should load a paginated datatable 7) Click into another tab, the tab should also load a paginated datatable 8) Click back into the first tab, the original table should persist and should not reload 9) Use the edit tools at the bottom of the page, they should work as expected 10) Click on the links in various table cells, they should redirect as expected 11) Use the actions buttons in the far right column, then should work as expected Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov> Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com> Sponsored by: Montgomery County Public Libraries Signed-off-by: krimsonkharne <philip.orr@posteo.de>
This is not working for me, I am getting 400 search, biblio, title="h" {"error":1,"error_code":"invalid_query"}
This does not seem correct: https://snipboard.io/W0kX5S.jpg Where are the other tabs?
Created attachment 182428 [details] [review] Bug 33430: DO NOT PUSH - data script
Created attachment 182429 [details] [review] Bug 33430: Add strings_map method This patch adds a strings_map method to allow the API to return the required authorised values needed for table displays
Created attachment 182430 [details] [review] Bug 33430: Update API specs to include required embeds
Created attachment 182431 [details] [review] Bug 33430: Pass search parameters to the template This patch removes the existing process intensive searches and moves the search logic to a single group query that returns the values that will be the counts in the tabs. It then formats the parameters for each type and passes them to the template for the datatables ajax requests
Created attachment 182432 [details] [review] Bug 33430: Enable datatables for suggestion tabs Test plan: 1) Either add loads of suggestions to your KTD or use the script in the data script patch uploaded to the bug 2) Navigate to Acquisitions > Suggestions 3) Depending on how many suggestions you have loaded (the script does 2000), the page will load very slowly 4) Apply patches 5) yarn build && restart_all 6) Hard refresh the page, the page should load much faster and the first tab should load a paginated datatable 7) Click into another tab, the tab should also load a paginated datatable 8) Click back into the first tab, the original table should persist and should not reload 9) Use the edit tools at the bottom of the page, they should work as expected 10) Click on the links in various table cells, they should redirect as expected 11) Use the actions buttons in the far right column, then should work as expected
Created attachment 182433 [details] [review] Bug 33430: (follow-up) Restore sorting to table
Created attachment 182434 [details] [review] Bug 33430: (follow-up) Fix the data filtering from the left menu
I'm testing. Looks great. I just found one error. Create a suggestion with library = "Any". Display suggestions with "Viewing suggestions for library" = "Any" Table crashes with : Uncaught TypeError: row.library is null
Also : Create a suggestion from OPAC with only title Look at suggestions table in staff interface It crashes with : Uncaught TypeError: row.manager is null
Created attachment 182478 [details] [review] Bug 33430: (follow-up) Handle undefined data in column renders
Created attachment 182480 [details] [review] Bug 33430: DO NOT PUSH - data script Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Created attachment 182481 [details] [review] Bug 33430: Add strings_map method This patch adds a strings_map method to allow the API to return the required authorised values needed for table displays Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Created attachment 182482 [details] [review] Bug 33430: Update API specs to include required embeds Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Created attachment 182483 [details] [review] Bug 33430: Pass search parameters to the template This patch removes the existing process intensive searches and moves the search logic to a single group query that returns the values that will be the counts in the tabs. It then formats the parameters for each type and passes them to the template for the datatables ajax requests Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Created attachment 182484 [details] [review] Bug 33430: Enable datatables for suggestion tabs Test plan: 1) Either add loads of suggestions to your KTD or use the script in the data script patch uploaded to the bug 2) Navigate to Acquisitions > Suggestions 3) Depending on how many suggestions you have loaded (the script does 2000), the page will load very slowly 4) Apply patches 5) yarn build && restart_all 6) Hard refresh the page, the page should load much faster and the first tab should load a paginated datatable 7) Click into another tab, the tab should also load a paginated datatable 8) Click back into the first tab, the original table should persist and should not reload 9) Use the edit tools at the bottom of the page, they should work as expected 10) Click on the links in various table cells, they should redirect as expected 11) Use the actions buttons in the far right column, then should work as expected Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Created attachment 182485 [details] [review] Bug 33430: (follow-up) Restore sorting to table Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Created attachment 182486 [details] [review] Bug 33430: (follow-up) Fix the data filtering from the left menu Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Created attachment 182487 [details] [review] Bug 33430: (follow-up) Handle undefined data in column renders Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Created attachment 182493 [details] [review] Bug 33430: DO NOT PUSH - data script Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com> Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Created attachment 182494 [details] [review] Bug 33430: Add strings_map method This patch adds a strings_map method to allow the API to return the required authorised values needed for table displays Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com> Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Created attachment 182495 [details] [review] Bug 33430: Update API specs to include required embeds Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com> Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Created attachment 182496 [details] [review] Bug 33430: Pass search parameters to the template This patch removes the existing process intensive searches and moves the search logic to a single group query that returns the values that will be the counts in the tabs. It then formats the parameters for each type and passes them to the template for the datatables ajax requests Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com> Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Created attachment 182497 [details] [review] Bug 33430: Enable datatables for suggestion tabs Test plan: 1) Either add loads of suggestions to your KTD or use the script in the data script patch uploaded to the bug 2) Navigate to Acquisitions > Suggestions 3) Depending on how many suggestions you have loaded (the script does 2000), the page will load very slowly 4) Apply patches 5) yarn build && restart_all 6) Hard refresh the page, the page should load much faster and the first tab should load a paginated datatable 7) Click into another tab, the tab should also load a paginated datatable 8) Click back into the first tab, the original table should persist and should not reload 9) Use the edit tools at the bottom of the page, they should work as expected 10) Click on the links in various table cells, they should redirect as expected 11) Use the actions buttons in the far right column, then should work as expected Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com> Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Created attachment 182498 [details] [review] Bug 33430: (follow-up) Restore sorting to table Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com> Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Created attachment 182499 [details] [review] Bug 33430: (follow-up) Fix the data filtering from the left menu Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com> Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Created attachment 182500 [details] [review] Bug 33430: (follow-up) Handle undefined data in column renders Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com> Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
1) Terminology + description: last member to modify the suggestion + library: + type: + - object + - "null" + description: branch the suggestion belongs to last user/patron to... library the suggestion... I wanted to fix these in a follow-up, but then found a more severe issue: 2) Anonymous suggestions break things To test: AnonymousPatron = 2 AnonSuggestions = Allow Add a suggestion from the OPAC without being logged in. The table of suggestions won't load: Organize by: Status Tab: Pending Uncaught TypeError: row.suggester is null
If this is an easy fix I'm inclined to try again Monday, but we need to make the decision that day.
3) Do dates sort correctly? I couldn't fully verify and I am not sure how the mark-up should like for datatables driven by API. I notice that before we had something like: <td data-order="2023-03-15" class="dt-type-date"> 15.03.2023 </td> Now we have just: <td>05.01.2025</td>
Created attachment 182577 [details] [review] Bug 33430: (follow-up) Fix API spec terminology
Created attachment 182578 [details] [review] Bug 33430: (follow-up) Prevent anonymous suggestions from breaking the table
I've fixed points 1 and 2, as far as I can tell the dates are sorting correctly for point 3 as well. I created a range of suggestions with different dates and using the column sorting was correctly ordering the dates