Bug 30738 - Forked CGI MARC import warnings are not logged
Summary: Forked CGI MARC import warnings are not logged
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: MARC Bibliographic record staging/import (show other bugs)
Version: Main
Hardware: All All
: P5 - low normal (vote)
Assignee: David Cook
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-05-12 03:46 UTC by David Cook
Modified: 2023-12-28 20:43 UTC (History)
2 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Trivial patch
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:
22.11.00


Attachments
Bug 30738: Log warnings for background MARC import (1.59 KB, patch)
2022-05-12 04:16 UTC, David Cook
Details | Diff | Splinter Review
Bug 30738: Log warnings for background MARC import (1.64 KB, patch)
2022-05-13 01:45 UTC, David Nind
Details | Diff | Splinter Review
Bug 30738: Log warnings for background MARC import (1.69 KB, patch)
2022-05-23 13:50 UTC, Nick Clemens (kidclamp)
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description David Cook 2022-05-12 03:46:29 UTC
While most of Koha can use Plack/PSGI these days, there are still a few CGI scripts like "manage-marc-import.pl".

This script forks itself in order to run in the background, and in so doing it closes STDOUT and STDERR to tell Apache not to control it anymore. (One day I hope to make this use the RabbitMQ, but that day is not today.)

Unfortunately, this means warnings get suppressed. I've noticed cases (usually with items) where a fatal error causes the import to fail (with the batch stuck in "Importing" with a 0% job progress bar) and there are no warnings in the logs! Very frustrating!

But we can use Koha::Logger to save the day...
Comment 1 David Cook 2022-05-12 04:16:31 UTC
Created attachment 134911 [details] [review]
Bug 30738: Log warnings for background MARC import

This change logs warnings from the CGI background MARC import.

Test plan:
0) Apply patch
1) Export a record with an item
2) Update the 952$f subfield to "Circulation" (any value over 10 chars)
3) Import the record and try to add the item
4) The import will stay stuck with "Importing" status and 0% job
progress
5) Check the /var/log/koha/kohadev/intranet-error.log file
and notice there's an error with the following text:
"Data too long for column 'coded_location_qualifier' at row 1"
Comment 2 David Nind 2022-05-13 01:45:20 UTC
Created attachment 134941 [details] [review]
Bug 30738: Log warnings for background MARC import

This change logs warnings from the CGI background MARC import.

Test plan:
0) Apply patch
1) Export a record with an item
2) Update the 952$f subfield to "Circulation" (any value over 10 chars)
3) Import the record and try to add the item
4) The import will stay stuck with "Importing" status and 0% job
progress
5) Check the /var/log/koha/kohadev/intranet-error.log file
and notice there's an error with the following text:
"Data too long for column 'coded_location_qualifier' at row 1"

Signed-off-by: David Nind <david@davidnind.com>
Comment 3 David Cook 2022-05-13 02:21:21 UTC
Thanks David!

I feel like this one would be useful to a lot of people...
Comment 4 Nick Clemens (kidclamp) 2022-05-23 13:50:13 UTC
Created attachment 135266 [details] [review]
Bug 30738: Log warnings for background MARC import

This change logs warnings from the CGI background MARC import.

Test plan:
0) Apply patch
1) Export a record with an item
2) Update the 952$f subfield to "Circulation" (any value over 10 chars)
3) Import the record and try to add the item
4) The import will stay stuck with "Importing" status and 0% job
progress
5) Check the /var/log/koha/kohadev/intranet-error.log file
and notice there's an error with the following text:
"Data too long for column 'coded_location_qualifier' at row 1"

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Comment 5 Nick Clemens (kidclamp) 2022-05-23 13:56:45 UTC
Note, this does what the title says, it logs warnings, errors like on 28152 are not logged
Comment 6 Tomás Cohen Arazi 2022-06-08 14:27:54 UTC
Pushed to master for 22.11.

Nice work everyone, thanks!