Bug 32266 - Make the late orders page render using the API
Summary: Make the late orders page render using the API
Status: Needs Signoff
Alias: None
Product: Koha
Classification: Unclassified
Component: Acquisitions (show other bugs)
Version: Main
Hardware: All All
: P5 - low enhancement
Assignee: Tomás Cohen Arazi (tcohen)
QA Contact: Testopia
URL:
Keywords:
Depends on: 39816 39830 39832
Blocks:
  Show dependency treegraph
 
Reported: 2022-11-18 13:12 UTC by Emmi Takkinen
Modified: 2025-05-12 12:59 UTC (History)
10 users (show)

See Also:
GIT URL:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:


Attachments
Bug 32266: Add new syspref DefaultLateOrderFilterDaysAgo (5.02 KB, patch)
2022-11-22 09:24 UTC, Emmi Takkinen
Details | Diff | Splinter Review
Bug 32266: Add new syspref DefaultLateOrderFilterDaysAgo (5.08 KB, patch)
2022-11-22 16:51 UTC, Amit Gupta
Details | Diff | Splinter Review
Bug 32266: Add new syspref DefaultLateOrderFilterDaysAgo (5.13 KB, patch)
2022-12-16 18:38 UTC, Kyle M Hall (khall)
Details | Diff | Splinter Review
Bug 32266: Add only_late filters to the endpoint (4.23 KB, patch)
2025-05-07 19:08 UTC, Tomás Cohen Arazi (tcohen)
Details | Diff | Splinter Review
Bug 32266: Add more embed options to orders and baskets (2.91 KB, patch)
2025-05-07 19:08 UTC, Tomás Cohen Arazi (tcohen)
Details | Diff | Splinter Review
Bug 32266: Make the late orders page use the API (47.01 KB, patch)
2025-05-07 19:08 UTC, Tomás Cohen Arazi (tcohen)
Details | Diff | Splinter Review
Bug 32266: (follow-up) Handle null basket.authorizer gracefully (2.25 KB, patch)
2025-05-12 12:59 UTC, Tomás Cohen Arazi (tcohen)
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Emmi Takkinen 2022-11-18 13:12:15 UTC
Almost all our library consortiums have large amounts of late orders. Smallest amount is around 2000, but largest is over 25000. Even with batch of 2000 orders, lateorders.pl takes over 40 seconds to open. Handling larger batches eventually leads to a Proxy error. 

One (quite easy imo) way to speed up fetching late orders is to add a syspref to set default value for late order filter "Order date: [] days ago".
Comment 1 Emmi Takkinen 2022-11-22 09:24:47 UTC
Created attachment 144145 [details] [review]
Bug 32266: Add new syspref DefaultLateOrderFilterDaysAgo

Some libraries might have large amounts of late orders that
slow down opening the lateorders.pl. E.g. patch of 2000
orders takes about 40 seconds to open, larger patches lead
to a proxy error.

This patch adds new syspref DefaultLateOrderFilterDaysAgo.
With this libraries can control how many days old late orders
should be for them to show in late orders listing.

To test:
1. Open browsers console (F12) and from there Network tab.
2. Navigate to Acquisitions -> Late Orders.
=> Take a note from browsers console how long it takes to open late orders listing.
3. Apply this patch, update you database and restart your services if needed.
4. Set some value to syspref DefaultLateOrderFilterDaysAgo (999 is max).
5. Load Late orders page again.
=> Loading page should now be significantly faster.
=> Only orders which

Sponsored-by: Koha-Suomi Oy
Comment 2 Amit Gupta 2022-11-22 16:51:07 UTC
Created attachment 144165 [details] [review]
Bug 32266: Add new syspref DefaultLateOrderFilterDaysAgo

To test:
1. Open browsers console (F12) and from there Network tab.
2. Navigate to Acquisitions -> Late Orders.
=> Take a note from browsers console how long it takes to open late orders listing.
3. Apply this patch, update you database and restart your services if needed.
4. Set some value to syspref DefaultLateOrderFilterDaysAgo (999 is max).
5. Load Late orders page again.

Nice enhancement.
Comment 3 Kyle M Hall (khall) 2022-12-16 18:38:03 UTC
Created attachment 144699 [details] [review]
Bug 32266: Add new syspref DefaultLateOrderFilterDaysAgo

Some libraries might have large amounts of late orders that
slow down opening the lateorders.pl. E.g. patch of 2000
orders takes about 40 seconds to open, larger patches lead
to a proxy error.

This patch adds new syspref DefaultLateOrderFilterDaysAgo.
With this libraries can control how many days old late orders
should be for them to show in late orders listing.

To test:
1. Open browsers console (F12) and from there Network tab.
2. Navigate to Acquisitions -> Late Orders.
=> Take a note from browsers console how long it takes to open late orders listing.
3. Apply this patch, update you database and restart your services if needed.
4. Set some value to syspref DefaultLateOrderFilterDaysAgo (999 is max).
5. Load Late orders page again.
=> Loading page should now be significantly faster.
=> Only orders which

Sponsored-by: Koha-Suomi Oy
Signed-off-by: Amit Gupta <amit.gupta@informaticsglobal.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 4 Tomás Cohen Arazi (tcohen) 2022-12-27 14:05:31 UTC
I understand where this is coming from, but we should really just copy and paste the API-based paginated orders table from parcel.tt and change some query parameters to match this use case...

I can help if needed.
Comment 5 Emmi Takkinen 2022-12-29 07:16:47 UTC
(In reply to Tomás Cohen Arazi from comment #4)
> I understand where this is coming from, but we should really just copy and
> paste the API-based paginated orders table from parcel.tt and change some
> query parameters to match this use case...
> 
> I can help if needed.

That is definitely a better solution in a long run. My plan was to first provide syspref as a simpler solution and then examine if we can optimize code in a different bug. 

I'll try adding that paginated table here and ask help if there are any problems.
Comment 6 Emmi Takkinen 2024-10-16 11:17:58 UTC
I haven't been able to work with this and probably won't be a long time. Removing myself from assigned so if someone is able to take a look at this feel free to do so.
Comment 7 Tomás Cohen Arazi (tcohen) 2025-05-07 19:08:12 UTC
Created attachment 182041 [details] [review]
Bug 32266: Add only_late filters to the endpoint
Comment 8 Tomás Cohen Arazi (tcohen) 2025-05-07 19:08:15 UTC
Created attachment 182042 [details] [review]
Bug 32266: Add more embed options to orders and baskets

This patch adds the ability to embed:

Orders:

* basket.vendor
* basket.authorizer+strings
* claims+count

Baskets:

* vendor
* authorizer+strings
Comment 9 Tomás Cohen Arazi (tcohen) 2025-05-07 19:08:19 UTC
Created attachment 182043 [details] [review]
Bug 32266: Make the late orders page use the API

This bug makes the late orders page render using the API.
Everything should work as usual.

To test:
0. Have a bunch of late orders, from different vendors with different
   dates
1. Check how the page works without the patches
2. Notice it works similar with the patches
3. Sign off :-D
Comment 10 Emmi Takkinen 2025-05-12 11:55:08 UTC
Page opens, but table doesn't load. In browsers console is a message:

>Uncaught TypeError: row.basket.authorizer is null
Maybe there needs to be a check if fetched row has authorisedby value in place, since it can be sometimes be NULL.
Comment 11 Tomás Cohen Arazi (tcohen) 2025-05-12 12:37:26 UTC
(In reply to Emmi Takkinen from comment #10)
> Page opens, but table doesn't load. In browsers console is a message:
> 
> >Uncaught TypeError: row.basket.authorizer is null
> Maybe there needs to be a check if fetched row has authorisedby value in
> place, since it can be sometimes be NULL.

Did you do:

```shell
 $ ktd --shell
k$ yarn build
k$ koha-plack --restart kohadev
```

before testing? I will double-check just in case, but it feels like I had that check in place.
Comment 12 Tomás Cohen Arazi (tcohen) 2025-05-12 12:59:11 UTC
Created attachment 182269 [details] [review]
Bug 32266: (follow-up) Handle null basket.authorizer gracefully

This patch makes the JS code handle the situation of
`aqbasket.authorisedby` being NULL.

I was only able to reproduce this by manually touching the DB, but
existing data might have this situation.

To test:
1. Have a few late orders
2. Pick a basket id
3. Run:
   $ ktd --shell
  k$ koha-mysql kohadev
   > UPDATE aqbasket SET authorisedby=NULL WHERE basketno=2;
   (in this case 2 was my chosen basket).
4. Visit the late orders page
=> FAIL: The table doesn't render correclty, the browser inspector says
something about authorizer being null
5. Apply this patch
6. Repeat 4
=> SUCCESS: The table renders correctly, the 'Library' column has an
empty string for the affected rows
7. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Comment 13 Tomás Cohen Arazi (tcohen) 2025-05-12 12:59:45 UTC
(In reply to Emmi Takkinen from comment #10)
> Page opens, but table doesn't load. In browsers console is a message:
> 
> >Uncaught TypeError: row.basket.authorizer is null
> Maybe there needs to be a check if fetched row has authorisedby value in
> place, since it can be sometimes be NULL.

Fixed!