Bug 33720

Summary: updatedatabase.pl should purge memcached
Product: Koha Reporter: Jake Deery <jake.deery>
Component: PackagingAssignee: Martin Renvoize (ashimema) <martin.renvoize>
Status: RESOLVED FIXED QA Contact: Marcel de Rooy <m.de.rooy>
Severity: minor    
Priority: P5 - low CC: jake.deery, jonathan.druart, m.de.rooy, martin.renvoize, mtj, nick
Version: Main   
Hardware: All   
OS: Linux   
Change sponsored?: --- Patch complexity: Trivial patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
23.11.00,23.05.02,22.11.08
Circulation function:
Attachments: Add memcached restart before updatedatabase.pl is called
Update the updatedatabase.pl script to flush caches before running
Bug 33720: Update the updatedatabase.pl script to flush caches before running
Bug 33720: Update the updatedatabase.pl script to flush caches before running
Bug 33720: Also flush in install
Bug 33720: Update the updatedatabase.pl script to flush caches before running
Bug 33720: Also flush in install

Description Jake Deery 2023-05-11 13:54:36 UTC
Hello community,

We have been caught out with a minor annoyance in the koha-upgrade-schema script (Debian package) - upon reloading a database from an SQL dump file (e.g. re-imaging a test server from a live system), the database upgrade appears to complete successfully, but in fact steps are missed.

We believe this is because memcached is keeping the software version cached as a value, as a restart of memcached seems to resolve the issue.

I attach a proposed solution; restarting memcached just before calling the updatedatabase.pl script.

I hope this makes sense, can can be backported for 22.11.xx.

Many thanks,
Jake
Comment 1 Jake Deery 2023-05-11 13:58:57 UTC
Created attachment 151083 [details] [review]
Add memcached restart before updatedatabase.pl is called

This patch adds a call to /etc/init.d/memcached so that when koha-upgrade-schema is called, the correct DB revision number is fetched, avoiding issues when loading & upgrading a SQL dump file
Comment 2 Jonathan Druart 2023-05-12 06:37:22 UTC
If we need a flush a can call bin/flush_memcached
Comment 3 Jonathan Druart 2023-05-12 06:39:38 UTC
Another solution would be to retrieve the value from the DB in updatedatabase.pl

74 my $original_version = C4::Context->preference("Version");
Comment 4 Jake Deery 2023-05-31 13:54:14 UTC
Hi Jonathan,

A fine point, well-raised. Flushing the cache from within updatedatabase.pl covers a wider range of install types, too.  I've attached another patch, this time for updatedatabase.pl. To test (assuming the software version is 22.1106000, please amend to suit): 

1. Set database to a historic version near to your own – SQL: UPDATE systempreferences SET value='22.1105012' WHERE variable='Version';
2. Run sudo koha-upgrade-schema test (or updatedatabase.pl, if you use dev installs) – notice how nothing happens
3. Apply patch
4. Rerun sudo koha-upgrade-schema test (or updatedatabase.pl, if you use dev installs) – notice how the upgrade proceeds from correct point (even if it fails due to duplicate tables / columns / keys)
5. Repeat steps 1 & 4 to validate

Many thanks,
Jake
Comment 5 Jake Deery 2023-05-31 13:55:06 UTC
Created attachment 151881 [details] [review]
Update the updatedatabase.pl script to flush caches before running
Comment 6 Jonathan Druart 2023-06-07 11:37:50 UTC
Created attachment 152096 [details] [review]
Bug 33720: Update the updatedatabase.pl script to flush caches before running
Comment 7 Jonathan Druart 2023-06-07 11:38:21 UTC
I've attached a git patch.
Comment 8 Jonathan Druart 2023-06-07 11:53:17 UTC
I think we should fix install.pl as well. The UI is not telling us the DB is behind if the syspref has been modified manually in DB.
Comment 9 Martin Renvoize (ashimema) 2023-06-29 08:21:00 UTC
Created attachment 152852 [details] [review]
Bug 33720: Update the updatedatabase.pl script to flush caches before running

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Comment 10 Martin Renvoize (ashimema) 2023-06-29 08:21:02 UTC
Created attachment 152853 [details] [review]
Bug 33720: Also flush in install
Comment 11 Marcel de Rooy 2023-06-29 09:49:39 UTC
Created attachment 152857 [details] [review]
Bug 33720: Update the updatedatabase.pl script to flush caches before running

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 12 Marcel de Rooy 2023-06-29 09:49:42 UTC
Created attachment 152858 [details] [review]
Bug 33720: Also flush in install

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 13 Tomás Cohen Arazi (tcohen) 2023-07-03 13:34:40 UTC
Pushed to master for 23.11.

Nice work everyone, thanks!
Comment 14 Martin Renvoize (ashimema) 2023-07-17 15:00:13 UTC
Thanks for all the hard work!

Pushed to 23.05.x for the next release
Comment 15 Matt Blenkinsop 2023-07-18 13:00:22 UTC
Nice work everyone!

Pushed to oldstable for 22.11.x