Bug 28352 - Errors in search_for_data_inconsistencies.pl relating to authorised values and frameworks
Summary: Errors in search_for_data_inconsistencies.pl relating to authorised values an...
Status: RESOLVED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Command-line Utilities (show other bugs)
Version: master
Hardware: All All
: P5 - low normal (vote)
Assignee: Nick Clemens
QA Contact: Marcel de Rooy
URL:
Keywords:
Depends on: 21466 27852
Blocks:
  Show dependency treegraph
 
Reported: 2021-05-13 19:40 UTC by Andrew Fuerste-Henry
Modified: 2021-10-26 00:43 UTC (History)
7 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Trivial patch
Text to go in the release notes:
Version(s) released in:
21.11.00,21.05.05,20.11.11


Attachments
Bug 28352: Only check authorised values mapped to DB fields (4.13 KB, patch)
2021-05-26 11:49 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 28352: Only check authorised values mapped to DB fields (4.29 KB, patch)
2021-08-04 07:30 UTC, Fridolin Somers
Details | Diff | Splinter Review
Bug 28352: Only check authorised values mapped to DB fields (4.38 KB, patch)
2021-10-01 09:54 UTC, Marcel de Rooy
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Andrew Fuerste-Henry 2021-05-13 19:40:09 UTC
When running search_for_data_inconsistencies.pl on 20.05 and later, I get the following errors:
Use of uninitialized value $tmp_kohafield in pattern match (m//) at /kohadevbox/koha/misc/maintenance/search_for_data_inconsistencies.pl line 151.
Use of uninitialized value $tmp_kohafield in substitution (s///) at /kohadevbox/koha/misc/maintenance/search_for_data_inconsistencies.pl line 154.
Can't call method "get_column" on an undefined value at /kohadevbox/koha/misc/maintenance/search_for_data_inconsistencies.pl line 157.

The script seems to be looking for something in authorised_values that it't not finding. I've verified that every authorised_value used in a marc_subfield_structure exists and has at least one value.
Comment 1 Nick Clemens 2021-05-26 11:49:22 UTC
Created attachment 121424 [details] [review]
Bug 28352: Only check authorised values mapped to DB fields

The errors reported seem to be caused by authorised values mapped to MARC fields
but not mapped to a koha field.

We should additionally make sure to check the Default framework

Also, adding comment to indicate we only check records with items, because we do

TO test:
1 - In a framework that is not the default map a MARC field to an authorised value, but not a koha field
2 - In SQL, force the kohafield to NULL for the mapping you just make
    UPDATE marc_subfield_structure SET kohafield = NULL WHERE frameworkcode='BKS' and authorised_value='HINGS_AS'
3 - perl misc/maintenance/search_for_data_inconsistencies.pl
4 - get the following errors:
Use of uninitialized value $tmp_kohafield in pattern match (m//) at /kohadevbox/koha/misc/maintenance/search_for_data_inconsistencies.pl line 151.
Use of uninitialized value $tmp_kohafield in substitution (s///) at /kohadevbox/koha/misc/maintenance/search_for_data_inconsistencies.pl line 154.
Can't call method "get_column" on an undefined value at /kohadevbox/koha/misc/maintenance/search_for_data_inconsistencies.pl line 157.
5 - Apply patch
6 - Repeat
7 - No more errors
Comment 2 Fridolin Somers 2021-08-04 07:30:43 UTC
Created attachment 123447 [details] [review]
Bug 28352: Only check authorised values mapped to DB fields

The errors reported seem to be caused by authorised values mapped to MARC fields
but not mapped to a koha field.

We should additionally make sure to check the Default framework

Also, adding comment to indicate we only check records with items, because we do

TO test:
1 - In a framework that is not the default map a MARC field to an authorised value, but not a koha field
2 - In SQL, force the kohafield to NULL for the mapping you just make
    UPDATE marc_subfield_structure SET kohafield = NULL WHERE frameworkcode='BKS' and authorised_value='HINGS_AS'
3 - perl misc/maintenance/search_for_data_inconsistencies.pl
4 - get the following errors:
Use of uninitialized value $tmp_kohafield in pattern match (m//) at /kohadevbox/koha/misc/maintenance/search_for_data_inconsistencies.pl line 151.
Use of uninitialized value $tmp_kohafield in substitution (s///) at /kohadevbox/koha/misc/maintenance/search_for_data_inconsistencies.pl line 154.
Can't call method "get_column" on an undefined value at /kohadevbox/koha/misc/maintenance/search_for_data_inconsistencies.pl line 157.
5 - Apply patch
6 - Repeat
7 - No more errors

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Comment 3 Fridolin Somers 2021-08-04 07:31:49 UTC
I was just about to open the same issue ;)
Thanks a lot Nick :D
Comment 4 Jonathan Druart 2021-08-04 09:01:45 UTC
Highlighed by bug 27852.
Comment 5 Jonathan Druart 2021-08-04 09:02:20 UTC
My fix was 
 143             next unless $kohafield and $av;

But yours is better.
Comment 6 Marcel de Rooy 2021-10-01 09:54:45 UTC
Created attachment 125599 [details] [review]
Bug 28352: Only check authorised values mapped to DB fields

The errors reported seem to be caused by authorised values mapped to MARC fields
but not mapped to a koha field.

We should additionally make sure to check the Default framework

Also, adding comment to indicate we only check records with items, because we do

TO test:
1 - In a framework that is not the default map a MARC field to an authorised value, but not a koha field
2 - In SQL, force the kohafield to NULL for the mapping you just make
    UPDATE marc_subfield_structure SET kohafield = NULL WHERE frameworkcode='BKS' and authorised_value='HINGS_AS'
3 - perl misc/maintenance/search_for_data_inconsistencies.pl
4 - get the following errors:
Use of uninitialized value $tmp_kohafield in pattern match (m//) at /kohadevbox/koha/misc/maintenance/search_for_data_inconsistencies.pl line 151.
Use of uninitialized value $tmp_kohafield in substitution (s///) at /kohadevbox/koha/misc/maintenance/search_for_data_inconsistencies.pl line 154.
Can't call method "get_column" on an undefined value at /kohadevbox/koha/misc/maintenance/search_for_data_inconsistencies.pl line 157.
5 - Apply patch
6 - Repeat
7 - No more errors

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 7 Jonathan Druart 2021-10-01 14:30:14 UTC
Pushed to master for 21.11, thanks to everybody involved!
Comment 8 Kyle M Hall 2021-10-08 13:52:49 UTC
Pushed to 21.05.x for 21.05.05
Comment 9 Fridolin Somers 2021-10-21 04:08:05 UTC
Pushed to 20.11.x for 20.11.11
Comment 10 Victor Grousset/tuxayo 2021-10-26 00:43:59 UTC
Not backported to oldoldstable (20.05.x). Feel free to ask if it's needed.