Bug 32266 - Make the late orders page render using the API
Summary: Make the late orders page render using the API
Status: Signed Off
Alias: None
Product: Koha
Classification: Unclassified
Component: Acquisitions (show other bugs)
Version: Main
Hardware: All All
: P5 - low enhancement
Assignee: Tomás Cohen Arazi (tcohen)
QA Contact: Testopia
URL:
Keywords:
Depends on: 39816 39830 39832
Blocks:
  Show dependency treegraph
 
Reported: 2022-11-18 13:12 UTC by Emmi Takkinen
Modified: 2025-05-14 05:42 UTC (History)
10 users (show)

See Also:
GIT URL:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
Circulation function:


Attachments
Bug 32266: Add new syspref DefaultLateOrderFilterDaysAgo (5.02 KB, patch)
2022-11-22 09:24 UTC, Emmi Takkinen
Details | Diff | Splinter Review
Bug 32266: Add new syspref DefaultLateOrderFilterDaysAgo (5.08 KB, patch)
2022-11-22 16:51 UTC, Amit Gupta
Details | Diff | Splinter Review
Bug 32266: Add new syspref DefaultLateOrderFilterDaysAgo (5.13 KB, patch)
2022-12-16 18:38 UTC, Kyle M Hall (khall)
Details | Diff | Splinter Review
Bug 32266: Add only_late filters to the endpoint (4.23 KB, patch)
2025-05-07 19:08 UTC, Tomás Cohen Arazi (tcohen)
Details | Diff | Splinter Review
Bug 32266: Add more embed options to orders and baskets (2.91 KB, patch)
2025-05-07 19:08 UTC, Tomás Cohen Arazi (tcohen)
Details | Diff | Splinter Review
Bug 32266: Make the late orders page use the API (47.01 KB, patch)
2025-05-07 19:08 UTC, Tomás Cohen Arazi (tcohen)
Details | Diff | Splinter Review
Bug 32266: (follow-up) Handle null basket.authorizer gracefully (2.25 KB, patch)
2025-05-12 12:59 UTC, Tomás Cohen Arazi (tcohen)
Details | Diff | Splinter Review
Bug 32266: Make the late orders page use the API (47.01 KB, patch)
2025-05-13 12:59 UTC, Tomás Cohen Arazi (tcohen)
Details | Diff | Splinter Review
Bug 32266: (follow-up) Handle null basket.authorizer gracefully (2.25 KB, patch)
2025-05-13 12:59 UTC, Tomás Cohen Arazi (tcohen)
Details | Diff | Splinter Review
Bug 32266: (follow-up) Add event listeners in correct context (5.39 KB, patch)
2025-05-13 12:59 UTC, Tomás Cohen Arazi (tcohen)
Details | Diff | Splinter Review
Bug 32266: Add only_late filters to the endpoint (4.29 KB, patch)
2025-05-14 05:40 UTC, Emmi Takkinen
Details | Diff | Splinter Review
Bug 32266: Add more embed options to orders and baskets (2.96 KB, patch)
2025-05-14 05:40 UTC, Emmi Takkinen
Details | Diff | Splinter Review
Bug 32266: Make the late orders page use the API (47.07 KB, patch)
2025-05-14 05:41 UTC, Emmi Takkinen
Details | Diff | Splinter Review
Bug 32266: (follow-up) Handle null basket.authorizer gracefully (2.31 KB, patch)
2025-05-14 05:41 UTC, Emmi Takkinen
Details | Diff | Splinter Review
Bug 32266: (follow-up) Add event listeners in correct context (5.45 KB, patch)
2025-05-14 05:42 UTC, Emmi Takkinen
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Emmi Takkinen 2022-11-18 13:12:15 UTC
Almost all our library consortiums have large amounts of late orders. Smallest amount is around 2000, but largest is over 25000. Even with batch of 2000 orders, lateorders.pl takes over 40 seconds to open. Handling larger batches eventually leads to a Proxy error. 

One (quite easy imo) way to speed up fetching late orders is to add a syspref to set default value for late order filter "Order date: [] days ago".
Comment 1 Emmi Takkinen 2022-11-22 09:24:47 UTC
Created attachment 144145 [details] [review]
Bug 32266: Add new syspref DefaultLateOrderFilterDaysAgo

Some libraries might have large amounts of late orders that
slow down opening the lateorders.pl. E.g. patch of 2000
orders takes about 40 seconds to open, larger patches lead
to a proxy error.

This patch adds new syspref DefaultLateOrderFilterDaysAgo.
With this libraries can control how many days old late orders
should be for them to show in late orders listing.

To test:
1. Open browsers console (F12) and from there Network tab.
2. Navigate to Acquisitions -> Late Orders.
=> Take a note from browsers console how long it takes to open late orders listing.
3. Apply this patch, update you database and restart your services if needed.
4. Set some value to syspref DefaultLateOrderFilterDaysAgo (999 is max).
5. Load Late orders page again.
=> Loading page should now be significantly faster.
=> Only orders which

Sponsored-by: Koha-Suomi Oy
Comment 2 Amit Gupta 2022-11-22 16:51:07 UTC
Created attachment 144165 [details] [review]
Bug 32266: Add new syspref DefaultLateOrderFilterDaysAgo

To test:
1. Open browsers console (F12) and from there Network tab.
2. Navigate to Acquisitions -> Late Orders.
=> Take a note from browsers console how long it takes to open late orders listing.
3. Apply this patch, update you database and restart your services if needed.
4. Set some value to syspref DefaultLateOrderFilterDaysAgo (999 is max).
5. Load Late orders page again.

Nice enhancement.
Comment 3 Kyle M Hall (khall) 2022-12-16 18:38:03 UTC
Created attachment 144699 [details] [review]
Bug 32266: Add new syspref DefaultLateOrderFilterDaysAgo

Some libraries might have large amounts of late orders that
slow down opening the lateorders.pl. E.g. patch of 2000
orders takes about 40 seconds to open, larger patches lead
to a proxy error.

This patch adds new syspref DefaultLateOrderFilterDaysAgo.
With this libraries can control how many days old late orders
should be for them to show in late orders listing.

To test:
1. Open browsers console (F12) and from there Network tab.
2. Navigate to Acquisitions -> Late Orders.
=> Take a note from browsers console how long it takes to open late orders listing.
3. Apply this patch, update you database and restart your services if needed.
4. Set some value to syspref DefaultLateOrderFilterDaysAgo (999 is max).
5. Load Late orders page again.
=> Loading page should now be significantly faster.
=> Only orders which

Sponsored-by: Koha-Suomi Oy
Signed-off-by: Amit Gupta <amit.gupta@informaticsglobal.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 4 Tomás Cohen Arazi (tcohen) 2022-12-27 14:05:31 UTC
I understand where this is coming from, but we should really just copy and paste the API-based paginated orders table from parcel.tt and change some query parameters to match this use case...

I can help if needed.
Comment 5 Emmi Takkinen 2022-12-29 07:16:47 UTC
(In reply to Tomás Cohen Arazi from comment #4)
> I understand where this is coming from, but we should really just copy and
> paste the API-based paginated orders table from parcel.tt and change some
> query parameters to match this use case...
> 
> I can help if needed.

That is definitely a better solution in a long run. My plan was to first provide syspref as a simpler solution and then examine if we can optimize code in a different bug. 

I'll try adding that paginated table here and ask help if there are any problems.
Comment 6 Emmi Takkinen 2024-10-16 11:17:58 UTC
I haven't been able to work with this and probably won't be a long time. Removing myself from assigned so if someone is able to take a look at this feel free to do so.
Comment 7 Tomás Cohen Arazi (tcohen) 2025-05-07 19:08:12 UTC
Created attachment 182041 [details] [review]
Bug 32266: Add only_late filters to the endpoint
Comment 8 Tomás Cohen Arazi (tcohen) 2025-05-07 19:08:15 UTC
Created attachment 182042 [details] [review]
Bug 32266: Add more embed options to orders and baskets

This patch adds the ability to embed:

Orders:

* basket.vendor
* basket.authorizer+strings
* claims+count

Baskets:

* vendor
* authorizer+strings
Comment 9 Tomás Cohen Arazi (tcohen) 2025-05-07 19:08:19 UTC
Created attachment 182043 [details] [review]
Bug 32266: Make the late orders page use the API

This bug makes the late orders page render using the API.
Everything should work as usual.

To test:
0. Have a bunch of late orders, from different vendors with different
   dates
1. Check how the page works without the patches
2. Notice it works similar with the patches
3. Sign off :-D
Comment 10 Emmi Takkinen 2025-05-12 11:55:08 UTC
Page opens, but table doesn't load. In browsers console is a message:

>Uncaught TypeError: row.basket.authorizer is null
Maybe there needs to be a check if fetched row has authorisedby value in place, since it can be sometimes be NULL.
Comment 11 Tomás Cohen Arazi (tcohen) 2025-05-12 12:37:26 UTC
(In reply to Emmi Takkinen from comment #10)
> Page opens, but table doesn't load. In browsers console is a message:
> 
> >Uncaught TypeError: row.basket.authorizer is null
> Maybe there needs to be a check if fetched row has authorisedby value in
> place, since it can be sometimes be NULL.

Did you do:

```shell
 $ ktd --shell
k$ yarn build
k$ koha-plack --restart kohadev
```

before testing? I will double-check just in case, but it feels like I had that check in place.
Comment 12 Tomás Cohen Arazi (tcohen) 2025-05-12 12:59:11 UTC
Created attachment 182269 [details] [review]
Bug 32266: (follow-up) Handle null basket.authorizer gracefully

This patch makes the JS code handle the situation of
`aqbasket.authorisedby` being NULL.

I was only able to reproduce this by manually touching the DB, but
existing data might have this situation.

To test:
1. Have a few late orders
2. Pick a basket id
3. Run:
   $ ktd --shell
  k$ koha-mysql kohadev
   > UPDATE aqbasket SET authorisedby=NULL WHERE basketno=2;
   (in this case 2 was my chosen basket).
4. Visit the late orders page
=> FAIL: The table doesn't render correclty, the browser inspector says
something about authorizer being null
5. Apply this patch
6. Repeat 4
=> SUCCESS: The table renders correctly, the 'Library' column has an
empty string for the affected rows
7. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Comment 13 Tomás Cohen Arazi (tcohen) 2025-05-12 12:59:45 UTC
(In reply to Emmi Takkinen from comment #10)
> Page opens, but table doesn't load. In browsers console is a message:
> 
> >Uncaught TypeError: row.basket.authorizer is null
> Maybe there needs to be a check if fetched row has authorisedby value in
> place, since it can be sometimes be NULL.

Fixed!
Comment 14 Emmi Takkinen 2025-05-13 07:23:59 UTC
Sorry but there are still some problems:

- Estimated delivery date and internal note editing opens to a new page, editing vendor note opens a pop-up. Without these patches they all open in a pop-up.
- Cannot open basket, url looks weird: 
> <a href="basket.pl?basketno=" 1976"="" title="10175">10175 (1976)</a>

But otherwise this works as expected.
Comment 15 Tomás Cohen Arazi (tcohen) 2025-05-13 11:59:45 UTC
(In reply to Emmi Takkinen from comment #14)
> Sorry but there are still some problems:
> 
> - Estimated delivery date and internal note editing opens to a new page,
> editing vendor note opens a pop-up. Without these patches they all open in a
> pop-up.

I think I'll just make them modals as in the basket page.

> - Cannot open basket, url looks weird: 
> > <a href="basket.pl?basketno=" 1976"="" title="10175">10175 (1976)</a>

This was a typo. Will fix right now.

> But otherwise this works as expected.

Great.
Comment 16 Tomás Cohen Arazi (tcohen) 2025-05-13 12:59:43 UTC
Created attachment 182377 [details] [review]
Bug 32266: Make the late orders page use the API

This bug makes the late orders page render using the API.
Everything should work as usual.

To test:
0. Have a bunch of late orders, from different vendors with different
   dates
1. Check how the page works without the patches
2. Notice it works similar with the patches
3. Sign off :-D
Comment 17 Tomás Cohen Arazi (tcohen) 2025-05-13 12:59:47 UTC
Created attachment 182378 [details] [review]
Bug 32266: (follow-up) Handle null basket.authorizer gracefully

This patch makes the JS code handle the situation of
`aqbasket.authorisedby` being NULL.

I was only able to reproduce this by manually touching the DB, but
existing data might have this situation.

To test:
1. Have a few late orders
2. Pick a basket id
3. Run:
   $ ktd --shell
  k$ koha-mysql kohadev
   > UPDATE aqbasket SET authorisedby=NULL WHERE basketno=2;
   (in this case 2 was my chosen basket).
4. Visit the late orders page
=> FAIL: The table doesn't render correclty, the browser inspector says
something about authorizer being null
5. Apply this patch
6. Repeat 4
=> SUCCESS: The table renders correctly, the 'Library' column has an
empty string for the affected rows
7. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Comment 18 Tomás Cohen Arazi (tcohen) 2025-05-13 12:59:50 UTC
Created attachment 182379 [details] [review]
Bug 32266: (follow-up) Add event listeners in correct context

As the datatable is redrawn asynchronously and on each interaction, the
initially defined event listeners have no effect. This yields a weird
rendering of the attribute edit form in a new page.

This patch makes the event listeners exist at the `$(#late_orders)`
level so they still apply to newly created cells on redrawing the
datatable.

A couple places get 'modernized' by using the arrow syntax for anonymous
functions. `e.target` is used instead of `$(this)` where it applies.

To test:
1. Without the patch, have some late orders
2. Try changing the delivery date or any of the notes
=> FAIL: You are sent to a new page
3. Apply this patch and reload
4. Repeat 2
=> SUCCESS: You are presented a nice modal instead (the original
behavior)
5. Change the values, verify the table is re-rendered and the values are
   correct
6. Sign off :-D
Comment 19 Emmi Takkinen 2025-05-14 05:40:31 UTC
Created attachment 182420 [details] [review]
Bug 32266: Add only_late filters to the endpoint

Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>
Comment 20 Emmi Takkinen 2025-05-14 05:40:59 UTC
Created attachment 182421 [details] [review]
Bug 32266: Add more embed options to orders and baskets

This patch adds the ability to embed:

Orders:

* basket.vendor
* basket.authorizer+strings
* claims+count

Baskets:

* vendor
* authorizer+strings

Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>
Comment 21 Emmi Takkinen 2025-05-14 05:41:29 UTC
Created attachment 182422 [details] [review]
Bug 32266: Make the late orders page use the API

This bug makes the late orders page render using the API.
Everything should work as usual.

To test:
0. Have a bunch of late orders, from different vendors with different
   dates
1. Check how the page works without the patches
2. Notice it works similar with the patches
3. Sign off :-D

Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>
Comment 22 Emmi Takkinen 2025-05-14 05:41:54 UTC
Created attachment 182423 [details] [review]
Bug 32266: (follow-up) Handle null basket.authorizer gracefully

This patch makes the JS code handle the situation of
`aqbasket.authorisedby` being NULL.

I was only able to reproduce this by manually touching the DB, but
existing data might have this situation.

To test:
1. Have a few late orders
2. Pick a basket id
3. Run:
   $ ktd --shell
  k$ koha-mysql kohadev
   > UPDATE aqbasket SET authorisedby=NULL WHERE basketno=2;
   (in this case 2 was my chosen basket).
4. Visit the late orders page
=> FAIL: The table doesn't render correclty, the browser inspector says
something about authorizer being null
5. Apply this patch
6. Repeat 4
=> SUCCESS: The table renders correctly, the 'Library' column has an
empty string for the affected rows
7. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>
Comment 23 Emmi Takkinen 2025-05-14 05:42:30 UTC
Created attachment 182424 [details] [review]
Bug 32266: (follow-up) Add event listeners in correct context

As the datatable is redrawn asynchronously and on each interaction, the
initially defined event listeners have no effect. This yields a weird
rendering of the attribute edit form in a new page.

This patch makes the event listeners exist at the `$(#late_orders)`
level so they still apply to newly created cells on redrawing the
datatable.

A couple places get 'modernized' by using the arrow syntax for anonymous
functions. `e.target` is used instead of `$(this)` where it applies.

To test:
1. Without the patch, have some late orders
2. Try changing the delivery date or any of the notes
=> FAIL: You are sent to a new page
3. Apply this patch and reload
4. Repeat 2
=> SUCCESS: You are presented a nice modal instead (the original
behavior)
5. Change the values, verify the table is re-rendered and the values are
   correct
6. Sign off :-D

Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>