We need to know which atomicupdates have been deployed to various test databases. This feature adds a database table to keep track of the deployed updates. There is also a cronjob to easily interface with the atomicupdate database log to revert entries and show them.
Created attachment 42112 [details] [review] Bug 14698 - AtomicUpdater - Keeps track of which updates have been applied to a database When deploying production databases to test environments, it is imperative to keep track of which database changes have been applied. When one is running, in production, features that require DB changes, and want to add more features which need more DB changes, keeping track of which updates have been applied is challenging. After several cycles of upgrade-deploy, with multiple atomicupdate-scripts piling to the atomicupdates/-directory, it is impossible to easily distinguish which updates have been applied and which are not. Rerunning the same update scripts again and again causes lots of noise in the upgrade log and hides real issues from being detected. Also repeatedly running upgrade scripts might cause bad side effects which are potentially hard to repair. This feature adds a script atomicupdate.pl which runs all atomicupdate/* scripts and .sqls and logs the event to the koha.atomicupdates-table. On subsequent runs of atomicupdate.pl, the already deployed upgrades are skipped, greatly reducing the unnecessary clutter and risk of upgrading. You can also remove existing log entries to allow rerunning the same upgrade script again, list all applied upgrades and show pending updates. Unit tests included.
Created attachment 42113 [details] Presentation about AtomicUpdate workflow for managing in-house features in production
Patch complexity is 'Large' because this change has many architectural connections
Created attachment 42385 [details] [review] Bug 14698 - AtomicUpdater - Keeps track of which updates have been applied to a database ... Also the order in which the atomicupdates need to be applied is important. This _updateorder is automatically generatable from the original Git repository.
Created attachment 42386 [details] [review] Bug 14698 - AtomicUpdater - Keeps track of which updates have been applied to a database
Created attachment 42395 [details] [review] Bug 14698 - AtomicUpdater - Keeps track of which updates have been applied to a database
Created attachment 42398 [details] [review] Bug 14698 - AtomicUpdater - Keeps track of which updates have been applied to a database
Created attachment 42426 [details] [review] Bug 14698 - AtomicUpdater - Keeps track of which updates have been applied to a database Fixed a missing PerlDependency. TEST PLAN: 1. Run "perl installer/data/mysql/atomicupdate.pl --help" 2. Read the code. 3. git bz apply some bugs which introduce atomicupdates, or cook your own 4. Deploy the atomicupdates using the atomicupdate.pl-script. 5. Repeat steps 3-4 to your heart's content. 6+. Understand the code and come up with improvement ideas!
Created attachment 42521 [details] [review] Bug 14698 - AtomicUpdater - Followup, add single atomicupdate-script execution.
Created attachment 42523 [details] [review] Bug 14698 - AtomicUpdater - Followup, add single atomicupdate-script execution. Add dry-run mode.
Applying: Bug 14698 - AtomicUpdater - Keeps track of which updates have been applied to a database /home/vagrant/kohaclone/.git/rebase-apply/patch:156: trailing whitespace. /home/vagrant/kohaclone/.git/rebase-apply/patch:493: trailing whitespace. /home/vagrant/kohaclone/.git/rebase-apply/patch:682: trailing whitespace. GetOptions( /home/vagrant/kohaclone/.git/rebase-apply/patch:765: trailing whitespace. fatal: sha1 information is lacking or useless (C4/Installer/PerlDependencies.pm). Repository lacks necessary blobs to fall back on 3-way merge. Cannot fall back to three-way merge. Patch failed at 0001 Bug 14698 - AtomicUpdater - Keeps track of which updates have been applied to a database
Hi! We are targeting to upgrade our Koha to master this summer and rebase this feature. Send me an email if you need this feature and we will share the latest code for you.
Created attachment 62141 [details] [review] Bug 14698 - AtomicUpdater - addAllAtomicUpdates This patch adds a new method Koha::AtomicUpdater->addAllAtomicUpdates that marks all pending atomicupdates as applied, but does not execute the database changes in them. This is required for fresh installations to avoid applying same updates after the installation. Binds this feature to web installer. To test: 1. prove t/db_dependent/Koha/AtomicUpdater.t
Created attachment 64404 [details] [review] Bug 14698: (follow-up) Koha::AtomicUpdater->find allow hashref as param Otherwise t/db_dependent/TestBuilder.t fails! To test: 1. prove t/db_dependent/Koha/AtomicUpdater.t 2. prove t/db_dependent/TestBuilder.t
Created attachment 102899 [details] [review] Bug 14698: Add needed Koha::Exceptions
Created attachment 102900 [details] [review] Bug 14698: AtomicUpdater - Keeps track of which updates have been applied to a database This patch includes latest code for atomicupdate.pl script. Tests are adjusted to work with TestBuilder and are no longer depended on Bug 13906.
Created attachment 102901 [details] [review] Bug 14698: AtomicUpdater - addAllAtomicUpdates Rebased patch.
Created attachment 102902 [details] [review] Bug 14698: AtomicUpdater - Bindings to updatedatabase.pl Instead of every time we execute updatedatabase.pl, we deploy AtomicUpdates smartly. Also tries it's best to parse the bug number from the updatedatabase.pl-entries and checks if that bug has already been applied via AtomicUpdater to prevent adding same DB changes twice.
Created attachment 102903 [details] [review] Bug 14698: AtomicUpdater - Support .perl atomicupdates as defined by skeleton.perl Koha community has introduced a template for developer atomicupdates, as defined in atomicupdates/skeleton.perl. This patch makes these updates Koha::AtomicUpdater compatible by redefining CheckVersion and SetVersion, and by passing $dbh into the file. To test: 1. prove t/db_dependent/Koha/AtomicUpdater.t
If the update entries are idempotent you won't face this problem.
I don't see how this would be useful for a production Koha. I could maybe see how it could be useful for a dev/test database, but that's not something that should be in mainline Koha I think. (Although following that logic, then atomic updates in general don't belong in mainline Koha. Awkward.) That said, I do wish that we'd improve our database upgrade methods...
*** This bug has been marked as a duplicate of bug 27880 ***