Bug 27466 - Update process failing for 20.06.00.023
Summary: Update process failing for 20.06.00.023
Status: RESOLVED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Installation and upgrade (command-line installer) (show other bugs)
Version: master
Hardware: PC Linux
: P5 - low blocker (vote)
Assignee: Jonathan Druart
QA Contact: Testopia
URL:
Keywords:
Depends on: 16371
Blocks:
  Show dependency treegraph
 
Reported: 2021-01-19 08:05 UTC by Manos PETRIDIS
Modified: 2021-02-17 19:35 UTC (History)
5 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Trivial patch
Text to go in the release notes:
The update 20.06.00.023 for adding options and reconfiguring the QuoteOfTheDay feature would fail. This patch makes sure that the update can be processed correctly.
Version(s) released in:
21.05.00,20.11.03


Attachments
A partial log of commands and errors encountered (17.47 KB, text/plain)
2021-01-19 08:05 UTC, Manos PETRIDIS
Details
Bug 27466: Don't use set_preference from updatedatabase.pl (2.22 KB, patch)
2021-01-20 10:28 UTC, Jonathan Druart
Details | Diff | Splinter Review
log4perl.conf from dev install (2.77 KB, text/plain)
2021-01-20 10:31 UTC, Jonathan Druart
Details
Bug 27466: Don't use set_preference from updatedatabase.pl (2.27 KB, patch)
2021-02-06 02:51 UTC, David Nind
Details | Diff | Splinter Review
Bug 27466: Don't use set_preference from updatedatabase.pl (2.33 KB, patch)
2021-02-07 11:26 UTC, Katrin Fischer
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Manos PETRIDIS 2021-01-19 08:05:48 UTC
Created attachment 115295 [details]
A partial log of commands and errors encountered

On Debian GNU/Linux 9.13 (stretch) with Linux 4.9.0-14-amd64 kernel.

Tried to upgrade koha to the latest release, using "apt-get upgrade".
As koha was held back, I tried "apt-get --with-new-pkgs upgrade". System then proceeded to the upgrade, installing any packages that were not originally required. However upgrade stops producing message "Configuration not defined at /usr/share/perl5/Log/Log4perl/Config.pm line 579."

...
Upgrade to 20.06.00.022 done [20:11:08]: Bug 21946 - Add parent type to itemtypes
Configuration not defined at /usr/share/perl5/Log/Log4perl/Config.pm line 579.
dpkg: error processing package koha-common (--configure):
 subprocess installed post-installation script returned error exit status 11
Processing triggers for systemd (232-25+deb9u12) ...
Errors were encountered while processing:
 koha-common
E: Sub-process /usr/bin/dpkg returned an error code (1)

Fuller details attached,
Comment 1 Jonathan Druart 2021-01-20 10:28:03 UTC
Created attachment 115417 [details] [review]
Bug 27466: Don't use set_preference from updatedatabase.pl

We must never use subroutines or methods from updatedatabase.pl!

Having it using C4::Context->set_preference will call
Koha::Config::Syspref->store then C4::Log::logaction and finally
Koha::Logger->get which will need the log4perl configured correctly
(which is not necessary the case when you upgrade).

Test plan:
git checkout ed3e4540d73c1c3009375ad339e09a2e676fa1bd (20.06.00.022)
reset_all
Set the value of QuoteOfTheDay to 0
git checkout bug_27466 (master + this patch)
updatedatabase
=> QuoteOfTheDay is an empty string
git checkout ed3e4540d73c1c3009375ad339e09a2e676fa1bd (20.06.00.022)
reset_all
Set the value of QuoteOfTheDay to 1
git checkout bug_27466 (master + this patch)
updatedatabase
=> QuoteOfTheDay is set to "opac"
Comment 2 Jonathan Druart 2021-01-20 10:31:05 UTC
Created attachment 115418 [details]
log4perl.conf from dev install
Comment 3 Jonathan Druart 2021-01-20 10:32:36 UTC
Thanks Manos for reporting this.

As a workaround (waiting for the next version) you can use the log4perl.conf file I have just attached, put it in /etc/koha/sites/$INSTANCE/log4perl.conf
Edit it and modify kohadev with your instance name.

That should work.
Comment 4 Jonathan Druart 2021-01-20 10:33:22 UTC
Caused by
  commit 6d7ff0d5040ca9759fee0609a6b7108773b81357
  Bug 16371: (follow-up) DB changes
Comment 5 Manos PETRIDIS 2021-01-21 08:05:39 UTC
Thank you Jonathan for your prompt and precise reply.
In my case is seems not to work, but it was heartwarming to see that someone has looked into the matter. Much appreciated!



root@koha:/etc/koha/sites/usmarc# pwd
/etc/koha/sites/usmarc

root@koha:/etc/koha/sites/usmarc# ls -la *.conf
-rw-rw-rw- 1 root root 2828 Jan 21 09:53 log4perl.conf

root@koha:/etc/koha/sites/usmarc# more *.conf
log4perl.logger.intranet = WARN, INTRANET
log4perl.appender.INTRANET=Log::Log4perl::Appender::File
log4perl.appender.INTRANET.filename=/var/log/koha/usmarc/intranet-error.log
log4perl.appender.INTRANET.mode=append
log4perl.appender.INTRANET.layout=PatternLayout
log4perl.appender.INTRANET.layout.ConversionPattern=[%d] [%p] %m %l %n
log4perl.appender.INTRANET.utf8=1

log4perl.logger.opac = WARN, OPAC
log4perl.appender.OPAC=Log::Log4perl::Appender::File
log4perl.appender.OPAC.filename=/var/log/koha/usmarc/opac-error.log
log4perl.appender.OPAC.mode=append
log4perl.appender.OPAC.layout=PatternLayout
log4perl.appender.OPAC.layout.ConversionPattern=[%d] [%p] %m %l %n
log4perl.appender.OPAC.utf8=1

log4perl.logger.z3950 = WARN, Z3950
log4perl.appender.Z3950=Log::Log4perl::Appender::File
log4perl.appender.Z3950.filename=/var/log/koha/usmarc/z3950-error.log
log4perl.appender.Z3950.mode=append
log4perl.appender.Z3950.layout=PatternLayout
log4perl.appender.Z3950.layout.ConversionPattern=[%d] [%p] %m %l %n
log4perl.appender.Z3950.utf8=1

log4perl.logger.api = WARN, API
log4perl.appender.API=Log::Log4perl::Appender::File
log4perl.appender.API.filename=/var/log/koha/usmarc/api-error.log
log4perl.appender.API.mode=append
log4perl.appender.API.layout=PatternLayout
log4perl.appender.API.layout.ConversionPattern=[%d] [%p] %m %l %n
log4perl.appender.API.utf8=1

log4perl.logger.sip = DEBUG, SIP
log4perl.appender.SIP=Log::Log4perl::Appender::File
log4perl.appender.SIP.filename=/var/log/koha/usmarc/sip.log
log4perl.appender.SIP.mode=append
log4perl.appender.SIP.layout=PatternLayout
log4perl.appender.SIP.layout.ConversionPattern=[%d] [%p] %m %l %n
log4perl.appender.SIP.utf8=1


log4perl.logger.plack-opac = WARN, PLACKOPAC
log4perl.appender.PLACKOPAC=Log::Log4perl::Appender::File
log4perl.appender.PLACKOPAC.filename=/var/log/koha/usmarc/plack-opac-error.log
log4perl.appender.PLACKOPAC.mode=append
log4perl.appender.PLACKOPAC.layout=PatternLayout
log4perl.appender.PLACKOPAC.layout.ConversionPattern=[%d] [%p] %m
log4perl.appender.PLACKOPAC.utf8=1

log4perl.logger.plack-api = WARN, PLACKAPI
log4perl.appender.PLACKAPI=Log::Log4perl::Appender::File
log4perl.appender.PLACKAPI.filename=/var/log/koha/usmarc/plack-api-error.log
log4perl.appender.PLACKAPI.mode=append
log4perl.appender.PLACKAPI.layout=PatternLayout
log4perl.appender.PLACKAPI.layout.ConversionPattern=[%d] [%p] %m
log4perl.appender.PLACKAPI.utf8=1

log4perl.logger.plack-intranet = WARN, PLACKINTRANET
log4perl.appender.PLACKINTRANET=Log::Log4perl::Appender::File
log4perl.appender.PLACKINTRANET.filename=/var/log/koha/usmarc/plack-intranet-err
or.log
log4perl.appender.PLACKINTRANET.mode=append
log4perl.appender.PLACKINTRANET.layout=PatternLayout
log4perl.appender.PLACKINTRANET.layout.ConversionPattern=[%d] [%p] %m
log4perl.appender.PLACKINTRANET.utf8=1

root@koha:/etc/koha/sites/usmarc# apt-get upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following packages were automatically installed and are no longer required:
  libpdf-api2-simple-perl libppi-perl libwww-youtube-download-perl
  libxml-treepp-perl tinymce
Use 'apt autoremove' to remove them.
The following packages have been kept back:
  koha-common libjson-validator-perl libmojolicious-perl
  libmojolicious-plugin-openapi-perl
0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.
1 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] y
Setting up koha-common (20.11.00-1) ...
Upgrading database schema for usmarc
Configuration not defined at /usr/share/perl5/Log/Log4perl/Config.pm line 579.
dpkg: error processing package koha-common (--configure):
 subprocess installed post-installation script returned error exit status 11
Errors were encountered while processing:
 koha-common
E: Sub-process /usr/bin/dpkg returned an error code (1)
root@koha:/etc/koha/sites/usmarc# 

As you can see, the problem persists.
Hope log above helps to clarify matters. 

kind regards,
Manos Petridis
Comment 6 Jonathan Druart 2021-01-21 09:09:37 UTC
Then maybe you are missing the entry in the koha-conf.xml file (inside "config")?

<log4perl_conf>/etc/koha/sites/usmarc/log4perl.conf</log4perl_conf>
Comment 7 Manos PETRIDIS 2021-01-21 22:55:50 UTC
Indeed, this addition fixed the problem. At first it seemed for some reason not to, and apt-get upgrade produced the same error message. After a reboot however, it proceeded fine(ish) in applying the upgrade. 

There was an error regarding Bug 25333, but koha is functional again.

Thanks again for your help!
Manos Petridis


root@koha:~# apt-get upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following packages were automatically installed and are no longer required:
  libcache-fastmmap-perl libpdf-api2-simple-perl libppi-perl
  libwww-youtube-download-perl libxml-treepp-perl tinymce
Use 'apt autoremove' to remove them.
The following packages have been kept back:
  libjson-validator-perl libmojolicious-perl
  libmojolicious-plugin-openapi-perl
0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.
1 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] y
Setting up koha-common (20.11.01-1) ...
Upgrading database schema for usmarc
Upgrade to 20.06.00.023 done [00:13:16]: Bug 16371 - Quote of the Day (QOTD) for the staff interface 
Upgrade to 20.06.00.024 done [00:13:17]: Bug 25867 - Update subfield descriptions for 952$a and 952$b
Upgrade to 20.06.00.025 done [00:13:17]: Bug  6725 - Adds PatronDuplicateMatchingAddFields system preference
Upgrade to 20.06.00.026 done [00:13:17]: Bug 19036 - Add accountlines.credit_number, account_credit_types.credit_number_enabled and syspref AutoCreditNumber
Upgrade to 20.06.00.027 done [00:13:17]: Bug  8732 - Add new BiblioItemtypeInfo to system preferences
Upgrade to 20.06.00.028 done [00:13:17]: Bug 25958 - Allow LongOverdue cron to exclude specified lost values
Upgrade to 20.06.00.029 done [00:13:17]: Bug 25534 - Add ability to send an email specifying a reason when canceling a hold
Upgrade to 20.06.00.030 done [00:13:17]: Bug 20057 - Add new system preference 'AutoApprovePatronProfileSettings'
Upgrade to 20.06.00.031 done [00:13:17]: Bug 22789 - Add non_priority column on reserves and old_reserves tables
Upgrade to 20.06.00.032 done [00:13:17]: Bug 19889 - Add exclude_from_local_holds_priority column to items, deleteditems and categories tables
Upgrade to 20.06.00.033 done [00:13:17]: Bug 21066 - Rename column opac_news.timestamp with published_on
						   - Add new column opac_news.updated_on
						   - Replace timestamp references in letters table
Upgrade to 20.06.00.034 done [00:13:17]: Bug 24197 - Add new system preference 'AddressForFailedOverdueNotices'
Upgrade to 20.06.00.035 done [00:13:17]: Bug 23682 - Add new system preference 'EdifactInvoiceImport'
Upgrade to 20.06.00.036 done [00:13:17]: Bug 20168 - Update OPACSearchForTitleIn to work with Bootstrap 4
Upgrade to 20.06.00.037 done [00:13:17]: Bug 23816 - Add min_password_length and require_strong_password columns in categories table
Upgrade to 20.06.00.038 done [00:13:18]: Bug 24807 - Add 'year' type to improve sorting behaviour
Upgrade to 20.06.00.039 done [00:13:18]: Bug 18958 - Add reserve_id to hold_fill_targets
Upgrade to 20.06.00.040 done [00:13:18]: Bug 26454 - Add system preference to set meta description for the OPAC
Upgrade to 20.06.00.041 done [00:13:18]: Bug 26268 - Remove items.paidfor field
Upgrade to 20.06.00.042 done [00:13:18]: Bug 25776 - Add letter.updated_on
Upgrade to 20.06.00.043 done [00:13:18]: Bug 25261 - Add CircConfirmItemParts syspref
Upgrade to 20.06.00.044 done [00:13:18]: Bug 22343 - Add SMTP configuration options
Upgrade to 20.06.00.045 done [00:13:18]: Bug 22417 - Add new table background_jobs
Upgrade to 20.06.00.046 done [00:13:18]: Bug 13535 - Add FK constraint on borrowernumber to alert table
Upgrade to 20.06.00.047 done [00:13:18]: Bug 23420 - Allow configuration of hidden fields on the suggestion form in OPAC
Upgrade to 20.06.00.048 done [00:13:18]: Bug 26529 - Remove blank default branch rules
Upgrade to 20.06.00.049 done [00:14:04]: Bug 26145 - Add the biblioimages.itemnumber column
						   - Rename table biblioimages with cover_images
Upgrade to 20.06.00.050 done [00:14:04]: Bug 19382 - Add ability to block guarantees based on fees owed by guarantor and other guarantee - new system preference 'NoIssuesChargeGuarantorsWithGuarantees'
Upgrade to 20.06.00.051 done [00:14:04]: Bug 12556 - Add new syspref HoldsNeedProcessingSIP
Upgrade to 20.06.00.052 done [00:14:04]: Bug 25460 - Update OAI set when adding/editing/deleting item records
Upgrade to 20.06.00.053 done [00:14:04]: Bug 26569 - Use gender neutral pronouns in system preference explanations
Upgrade to 20.06.00.054 done [00:14:04]: Bug 25596 - Add OVERPAYMENT credit type
Upgrade to 20.06.00.055 done [00:14:05]: Bug 18050 - Add FK constraint on aqbudgets.budget_period_id
Upgrade to 20.06.00.056 done [00:14:07]: Bug 26853 - Update import_biblios columns and indexes
Upgrade to 20.06.00.057 done [00:14:07]: Bug 26638 - Add missing system preference ArticleRequestsMandatoryFieldsItemOnly
DBD::mysql::db do failed: Cannot delete or update a parent row: a foreign key constraint fails (`koha_usmarc`.`letter`, CONSTRAINT `message_transport_type_fk` FOREIGN KEY (`message_transport_type`) REFERENCES `message_transport_types` (`message_transport_type`)) [for Statement "
        UPDATE message_transport_types SET message_transport_type = "itiva" WHERE message_transport_type = "phone"
    "] at /usr/share/koha/intranet/cgi-bin/installer/data/mysql/updatedatabase.pl line 23192.
Upgrade to 20.06.00.058 done [00:14:07]: Bug 25333 - Change message transport type for Talking Tech from "phone" to "itiva"
Upgrade to 20.06.00.059 done [00:14:07]: Bug 19482 - Add mandatory column to search_field for ES mapping
Upgrade to 20.06.00.060 done [00:14:08]: Bug 25334 - Add generic 'phone' message transport type
Upgrade to 20.06.00.061 done [00:14:08]: Bug 24412 - Attach waiting reserve to desk
Upgrade to 20.06.00.062 done [00:14:08]: Bug 23091 - Update refund rules
Upgrade to 20.06.00.063 done [00:14:08]: Bug 14866 - Add decreaseloanholds circulation rule
Upgrade to 20.06.00.064 done [00:14:08]: Bug 24603 - Add CANCELLATION credit_type_code
Upgrade to 20.06.00.065 done [00:14:08]: Bug 23916 - Add new [old_]issues.issuer DB fields
						   - Add new syspref RecordStaffUserOnCheckout
Upgrade to 20.06.00.066 done [00:14:09]: Bug 22818 - Add ILL notices
Upgrade to 20.06.00.067 done [00:14:09]: Bug 20936 - Add new system preference OPACHoldsHistory
Upgrade to 20.06.00.068 done [00:14:09]: Bug 23019 - Add import_batch_profiles table and profile_id column in import_batches
Upgrade to 20.06.00.069 done [00:14:09]: Bug 24083 - Add circulation_rules 'unseen_renewals_allowed'
						   - Add issues.unseen_renewals & old_issues.unseen_renewals)
						   - Add new system preference UnseenRenewals
Upgrade to 20.11.00.000 done [00:14:09]: Koha 20.11.00 release
Upgrade to 20.11.01.000 done [00:14:09]: Koha 20.11.01 release
The following plugins have been enabled:
  amqp_client
  rabbitmq_stomp

Applying plugin configuration to rabbit@koha... started 2 plugins.
W: APT had planned for dpkg to do more than it reported back (0 vs 4).
   Affected packages: koha-common:amd64
root@koha:~#
Comment 8 Jonathan Druart 2021-01-22 08:36:34 UTC
Please open a new bug report to report the error and link it to bug 25333.
Comment 9 Manos PETRIDIS 2021-01-23 10:28:47 UTC
(In reply to Jonathan Druart from comment #8)
> Please open a new bug report to report the error and link it to bug 25333.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27534

Hope I was correct in classifying it as belonging to the same component as the original bug. 

king regards,
Manos Petridis
Comment 10 Jonathan Druart 2021-01-29 10:59:25 UTC
Manos, did you test the patch?
Comment 11 David Nind 2021-02-06 02:51:09 UTC
Created attachment 116439 [details] [review]
Bug 27466: Don't use set_preference from updatedatabase.pl

We must never use subroutines or methods from updatedatabase.pl!

Having it using C4::Context->set_preference will call
Koha::Config::Syspref->store then C4::Log::logaction and finally
Koha::Logger->get which will need the log4perl configured correctly
(which is not necessary the case when you upgrade).

Test plan:
git checkout ed3e4540d73c1c3009375ad339e09a2e676fa1bd (20.06.00.022)
reset_all
Set the value of QuoteOfTheDay to 0
git checkout bug_27466 (master + this patch)
updatedatabase
=> QuoteOfTheDay is an empty string
git checkout ed3e4540d73c1c3009375ad339e09a2e676fa1bd (20.06.00.022)
reset_all
Set the value of QuoteOfTheDay to 1
git checkout bug_27466 (master + this patch)
updatedatabase
=> QuoteOfTheDay is set to "opac"

Signed-off-by: David Nind <david@davidnind.com>
Comment 12 David Nind 2021-02-06 02:56:29 UTC
My testing notes.

Test plan (numbered):
1. git checkout ed3e4540d73c1c3009375ad339e09a2e676fa1bd (20.06.00.022)
2. reset_all
3. Set the value of QuoteOfTheDay to 0
4. git checkout bug_27466 (master + this patch)
5. updatedatabase
   => QuoteOfTheDay is an empty string
6. git checkout ed3e4540d73c1c3009375ad339e09a2e676fa1bd (20.06.00.022)
7. reset_all
8. Set the value of QuoteOfTheDay to 1
9. git checkout bug_27466 (master + this patch)
10. updatedatabase
    => QuoteOfTheDay is set to "opac"

Notes:
- Used koha-testing-docker
- Step 3: 
  . koha-mysql kohadev
  . SELECT value FROM systempreferences WHERE variable='QuoteOfTheDay'; 
  . was already 0 (QuoteOfTheDay not enabled)
- Step 4: 
  . git checkout master
  . git checkout -b bz27466
  . git bz apply 27466
- Step 5:
  . same as step 3
  . there is no value, that is, it is empty
- Step 8:
  . log in to the staff interface and enable the QuoteOfTheDay system preference
- Step 9: git checkout -b bz27466
- Step 10:
  . koha-mysql kohadev
  . SELECT value FROM systempreferences WHERE variable='QuoteOfTheDay'; 
  . QuoteOfTheDay is set to "opac"!
Comment 13 Katrin Fischer 2021-02-07 11:26:15 UTC
Created attachment 116460 [details] [review]
Bug 27466: Don't use set_preference from updatedatabase.pl

We must never use subroutines or methods from updatedatabase.pl!

Having it using C4::Context->set_preference will call
Koha::Config::Syspref->store then C4::Log::logaction and finally
Koha::Logger->get which will need the log4perl configured correctly
(which is not necessary the case when you upgrade).

Test plan:
git checkout ed3e4540d73c1c3009375ad339e09a2e676fa1bd (20.06.00.022)
reset_all
Set the value of QuoteOfTheDay to 0
git checkout bug_27466 (master + this patch)
updatedatabase
=> QuoteOfTheDay is an empty string
git checkout ed3e4540d73c1c3009375ad339e09a2e676fa1bd (20.06.00.022)
reset_all
Set the value of QuoteOfTheDay to 1
git checkout bug_27466 (master + this patch)
updatedatabase
=> QuoteOfTheDay is set to "opac"

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Comment 14 Jonathan Druart 2021-02-08 13:57:47 UTC
Pushed to master for 21.05, thanks to everybody involved!
Comment 15 Fridolin Somers 2021-02-12 12:47:38 UTC
Pushed to 20.11.x for 20.11.03
Comment 16 Andrew Fuerste-Henry 2021-02-17 19:35:15 UTC
Missing dependency, not backported to 20.05