Bug 39930

Summary: Saved configuration states on tables are lost overnight
Product: Koha Reporter: Jason Robb <jrobb>
Component: Staff interfaceAssignee: Jonathan Druart <jonathan.druart>
Status: Pushed to oldstable --- QA Contact: Testopia <testopia>
Severity: major    
Priority: P5 - low CC: ahernandez, andrew, baptiste.wojtkowski, carthur, david, fridolin.somers, gmcharlt, jonathan.druart, ludovic.julien, mathsabypro, mteal, roman.dolny
Version: MainKeywords: rel_24_11_candidate
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40753
GIT URL: Change sponsored?: ---
Patch complexity: --- Documentation contact:
Documentation submission: Text to go in the release notes:
Version(s) released in:
25.11.00,25.05.04,24.11.10
Circulation function:
Bug Depends on: 33484    
Bug Blocks:    
Attachments: Bug 39930: Fix SaveState on the orders table
Bug 39930: Prevent DT's state to be lost overnight
Bug 39930: Prevent DT's state to be lost overnight
Bug 39930: Prevent DT's state to be lost overnight

Description Jason Robb 2025-05-16 21:36:06 UTC
This is hard to replicate but I did come across it on main in ktd.

When "Save configuration state on page change" is enabled for a table, after an amount of time or some undetermined action, the table settings seem to go wonky.

On the issues table at circ/circulation.pl for example, the grouping breaks and the sort_order and todays_or_previous_checkouts get revealed where they should be perma-hidden:

Table showing columns that shouldn't be there: https://snipboard.io/TYL3Oe.jpg
Table settings for that table: https://snipboard.io/kU7Tiw.jpg
Column settings from local storage: https://snipboard.io/6aS8e4.jpg

The steps that led to this were:
1. Set AlwaysLoadCheckoutsTable syspref to Do
2. Check out an item (39999000018516) to a patron (42)
3. Close the tab for a while (I didn't log out, just closed then reopened a few hours later)
4. Pull up the patron, extra mysterious columns have appeared

This isn't isolated to the checkouts table, as we've come across it with saved reports and other tables that get saved states as well.

Unchecking the save state box seems to avoid the problem.
Comment 1 mteal 2025-06-24 17:34:51 UTC
We saw this in Acquisitions. When we search for Orders for a particular vendor, it shows a list of baskets. If you click on and open a basket, it correctly displays the orders table with the columns reflecting what is set up in Configuration.
If you back arrow to get back to your list and select another basket, the columns remain correct.
However, while you have a basket open, if you click on "Reopen basket", the Orders table displays with ALL the columns, even the ones selected to hide. From that point on, it continues to show all the columns no matter what baskets I pull up. You have to log out of Koha and back in to get the columns to reset back to what's in Configuration. 
Unchecking Save configuration state on page change prevents the problem from happening.
Comment 2 Jonathan Druart 2025-07-24 13:29:52 UTC
I do recreate the problem on the orders table.

When a basket is closed we lost 2 columns, DataTables cannot restore the state we previously stored (in localStorage) and use a default state: all columns displayed.

I have not tried what Jason described on comment 0, but I guess it's a similar problem.

It seems to be not trivial, at first glance I don't think we will be able to fix this globally, we will have to adjust the tables one by one. We will certainly one bug report per table.

I was not aware of this bug, sorry to have missed it!
Comment 3 Jonathan Druart 2025-07-25 10:27:39 UTC
Created attachment 184625 [details] [review]
Bug 39930: Fix SaveState on the orders table

Test plan:
Open and close a basket
Comment 4 Jonathan Druart 2025-07-25 10:30:41 UTC
On the orders table the issue appears because we show/hide columns depending on some conditions (sysprefs or if the basket is closed/opened). DataTables cannot correctly restore the state because the number of columns stored in localStorage is different than the one we have one the table.

The orders table is, I think, fixed by this patch.

However I don't recreate the problem with the checkouts or reports. Could do try in a sandbox and provide a clear test plan to recreate the problem? Or give me another table where the problem appears?
Comment 5 Andrew Fuerste-Henry 2025-08-11 17:45:26 UTC
Jason, would you be amenable to re-focusing this bug on just the acq orders table issue that Jonathan has recreated and patched?
Comment 6 Jason Robb 2025-08-12 14:52:33 UTC
(In reply to Andrew Fuerste-Henry from comment #5)
> Jason, would you be amenable to re-focusing this bug on just the acq orders
> table issue that Jonathan has recreated and patched?

It's cool with me if it helps the fix for the orders table move along.

I did go through the steps I listed in my original bug report with an up-to-date ktd and it's still happening for me on the checkouts table. I did steps 1-3, then left it overnight and when I pulled the table up in the morning the extra columns had appeared.

Not sure if its related but I noticed in the DataTables documentation that the default value for stateDuration is 7200 so maybe 2 hours is the minimum amount of time that needs to elapse between steps 3 and 4 to see the issue?
Comment 7 Jonathan Druart 2025-09-02 14:27:03 UTC
(In reply to Jason Robb from comment #6)
> (In reply to Andrew Fuerste-Henry from comment #5)
> > Jason, would you be amenable to re-focusing this bug on just the acq orders
> > table issue that Jonathan has recreated and patched?
> 
> It's cool with me if it helps the fix for the orders table move along.
> 
> I did go through the steps I listed in my original bug report with an
> up-to-date ktd and it's still happening for me on the checkouts table. I did
> steps 1-3, then left it overnight and when I pulled the table up in the
> morning the extra columns had appeared.
> 
> Not sure if its related but I noticed in the DataTables documentation that
> the default value for stateDuration is 7200 so maybe 2 hours is the minimum
> amount of time that needs to elapse between steps 3 and 4 to see the issue?

Good catch, I have tried to set it to 60s and I do recreate the problem easily now.
I will be back here soon.

I think the only solution would be to extend the stateDuration value.
Comment 8 Jonathan Druart 2025-09-04 10:49:40 UTC
(In reply to Jonathan Druart from comment #3)
> Created attachment 184625 [details] [review] [review]
> Bug 39930: Fix SaveState on the orders table
> 
> Test plan:
> Open and close a basket

Moved to bug 40753.
Comment 9 Jonathan Druart 2025-09-04 10:56:35 UTC
Created attachment 186139 [details] [review]
Bug 39930: Prevent DT's state to be lost overnight

The default value for stateDuration is 7200 (2h). Which means DataTables
will ignore the saved state if reused after this delay.

Additionally is causes problem on some tables where the default state is
not restored and all columns are displayed.

From https://datatables.net/reference/option/stateDuration
```
Please note that the value is given in seconds. The value 0 is a special value as it indicates that the state can be stored and retrieved indefinitely with no time limit.
```

We are going to set it to 0 so that the state is only lost/reset after a
logout

Test plan:
Ideally you can should edit this file and set saveState to 60 (seconds),
so it's easy to test.
Do not apply this patch and confirm that the state is not restored after
either 60s or 2h (if you modified the file or not).
It is a problem on the checkout list table as extra columns (like
"Today's checkouts") are displayed.

With this patch applied the state should be restored correctly.
If you logout the state will be lost however.

Ideally we might want to store user's preferences on the server.
Comment 10 Lin Wei 2025-09-04 19:58:01 UTC
Created attachment 186182 [details] [review]
Bug 39930: Prevent DT's state to be lost overnight

The default value for stateDuration is 7200 (2h). Which means DataTables
will ignore the saved state if reused after this delay.

Additionally is causes problem on some tables where the default state is
not restored and all columns are displayed.

From https://datatables.net/reference/option/stateDuration
```
Please note that the value is given in seconds. The value 0 is a special value as it indicates that the state can be stored and retrieved indefinitely with no time limit.
```

We are going to set it to 0 so that the state is only lost/reset after a
logout

Test plan:
Ideally you can should edit this file and set saveState to 60 (seconds),
so it's easy to test.
Do not apply this patch and confirm that the state is not restored after
either 60s or 2h (if you modified the file or not).
It is a problem on the checkout list table as extra columns (like
"Today's checkouts") are displayed.

With this patch applied the state should be restored correctly.
If you logout the state will be lost however.

Ideally we might want to store user's preferences on the server.

Signed-off-by: Ludovic Julien <ludovic.julien@inLibro.com>
Comment 11 Nick Clemens (kidclamp) 2025-09-05 16:03:20 UTC
Created attachment 186238 [details] [review]
Bug 39930: Prevent DT's state to be lost overnight

The default value for stateDuration is 7200 (2h). Which means DataTables
will ignore the saved state if reused after this delay.

Additionally is causes problem on some tables where the default state is
not restored and all columns are displayed.

From https://datatables.net/reference/option/stateDuration
```
Please note that the value is given in seconds. The value 0 is a special value as it indicates that the state can be stored and retrieved indefinitely with no time limit.
```

We are going to set it to 0 so that the state is only lost/reset after a
logout

Test plan:
Ideally you can should edit this file and set saveState to 60 (seconds),
so it's easy to test.
Do not apply this patch and confirm that the state is not restored after
either 60s or 2h (if you modified the file or not).
It is a problem on the checkout list table as extra columns (like
"Today's checkouts") are displayed.

With this patch applied the state should be restored correctly.
If you logout the state will be lost however.

Ideally we might want to store user's preferences on the server.

Signed-off-by: Ludovic Julien <ludovic.julien@inLibro.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 12 Lucas Gass (lukeg) 2025-09-05 21:17:20 UTC
Nice work everyone!

Pushed to main for 25.11
Comment 13 Paul Derscheid 2025-09-20 08:58:01 UTC
Nice work everyone!

Pushed to 25.05.x
Comment 14 Mathieu Saby 2025-10-02 09:29:24 UTC
It is affecting our 24.11. Could it be pushed to 24.11.x ?
Comment 15 Fridolin Somers 2025-10-02 15:55:02 UTC
On it
Comment 16 Fridolin Somers 2025-10-03 07:03:47 UTC
Pushed to 24.11.x for 24.11.10