Bug 30738

Summary: Forked CGI MARC import warnings are not logged
Product: Koha Reporter: David Cook <dcook>
Component: MARC Bibliographic record staging/importAssignee: David Cook <dcook>
Status: CLOSED FIXED QA Contact: Testopia <testopia>
Severity: normal    
Priority: P5 - low CC: nick, tomascohen
Version: Main   
Hardware: All   
OS: All   
See Also: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30739
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28152
Change sponsored?: --- Patch complexity: Trivial patch
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
22.11.00
Attachments: Bug 30738: Log warnings for background MARC import
Bug 30738: Log warnings for background MARC import
Bug 30738: Log warnings for background MARC import

Description David Cook 2022-05-12 03:46:29 UTC
While most of Koha can use Plack/PSGI these days, there are still a few CGI scripts like "manage-marc-import.pl".

This script forks itself in order to run in the background, and in so doing it closes STDOUT and STDERR to tell Apache not to control it anymore. (One day I hope to make this use the RabbitMQ, but that day is not today.)

Unfortunately, this means warnings get suppressed. I've noticed cases (usually with items) where a fatal error causes the import to fail (with the batch stuck in "Importing" with a 0% job progress bar) and there are no warnings in the logs! Very frustrating!

But we can use Koha::Logger to save the day...
Comment 1 David Cook 2022-05-12 04:16:31 UTC
Created attachment 134911 [details] [review]
Bug 30738: Log warnings for background MARC import

This change logs warnings from the CGI background MARC import.

Test plan:
0) Apply patch
1) Export a record with an item
2) Update the 952$f subfield to "Circulation" (any value over 10 chars)
3) Import the record and try to add the item
4) The import will stay stuck with "Importing" status and 0% job
progress
5) Check the /var/log/koha/kohadev/intranet-error.log file
and notice there's an error with the following text:
"Data too long for column 'coded_location_qualifier' at row 1"
Comment 2 David Nind 2022-05-13 01:45:20 UTC
Created attachment 134941 [details] [review]
Bug 30738: Log warnings for background MARC import

This change logs warnings from the CGI background MARC import.

Test plan:
0) Apply patch
1) Export a record with an item
2) Update the 952$f subfield to "Circulation" (any value over 10 chars)
3) Import the record and try to add the item
4) The import will stay stuck with "Importing" status and 0% job
progress
5) Check the /var/log/koha/kohadev/intranet-error.log file
and notice there's an error with the following text:
"Data too long for column 'coded_location_qualifier' at row 1"

Signed-off-by: David Nind <david@davidnind.com>
Comment 3 David Cook 2022-05-13 02:21:21 UTC
Thanks David!

I feel like this one would be useful to a lot of people...
Comment 4 Nick Clemens (kidclamp) 2022-05-23 13:50:13 UTC
Created attachment 135266 [details] [review]
Bug 30738: Log warnings for background MARC import

This change logs warnings from the CGI background MARC import.

Test plan:
0) Apply patch
1) Export a record with an item
2) Update the 952$f subfield to "Circulation" (any value over 10 chars)
3) Import the record and try to add the item
4) The import will stay stuck with "Importing" status and 0% job
progress
5) Check the /var/log/koha/kohadev/intranet-error.log file
and notice there's an error with the following text:
"Data too long for column 'coded_location_qualifier' at row 1"

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 5 Nick Clemens (kidclamp) 2022-05-23 13:56:45 UTC
Note, this does what the title says, it logs warnings, errors like on 28152 are not logged
Comment 6 Tomás Cohen Arazi 2022-06-08 14:27:54 UTC
Pushed to master for 22.11.

Nice work everyone, thanks!