Bug 22013 - Command line staging: Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Summary: Command line staging: Use of uninitialized value $tag in hash element at /usr...
Status: NEW
Alias: None
Product: Koha
Classification: Unclassified
Component: MARC Authority data support (show other bugs)
Version: 18.11
Hardware: All All
: P5 - low enhancement (vote)
Assignee: Bugs List
QA Contact: Testopia
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-12-17 17:31 UTC by Simon Hohl
Modified: 2023-09-25 06:39 UTC (History)
2 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Documentation contact:
Documentation submission:
Text to go in the release notes:
Version(s) released in:


Attachments
MARC File containing geographical authority data (23.89 KB, application/marc)
2018-12-17 17:31 UTC, Simon Hohl
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Simon Hohl 2018-12-17 17:31:18 UTC
Created attachment 83307 [details]
MARC File containing geographical authority data

When using stage_file.pl to import authority data (see attachment), I get the warning:

Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in regexp compilation at /usr/share/perl5/MARC/Record.pm line 206.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 207.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.

That's 28 warnings for 31 records, but I am unable to decide which cause the warning and which do not. Anybody able to find out why a pattern?

Cheers
Comment 1 Jonathan Druart 2018-12-17 19:59:38 UTC
Which exact command are you running?

% perl misc/stage_file.pl --file gazetteer_authority.mrc --authorities
... staging MARC records -- please wait
... finished staging MARC records

MARC record staging report
------------------------------------
Input file:                 gazetteer_authority.mrc
Record type:                auth
Number of input records:    31
Number of valid records:    31
Number of invalid records:  0
Incoming records not matched against existing records (--match option not supplied)

Batch number assigned:  3


% perl misc/commit_file.pl --batch-number=3
... importing MARC records -- please wait
... finished importing MARC records

MARC record import report
----------------------------------------
Batch number:                    3
Number of new records added:     31
Number of records replaced:      0
Number of records ignored:       0
Number of items added:           0
Number of items replaced:        0
Number of items ignored:         0

Note: an item is ignored if its barcode is a
duplicate of one already in the database.
Comment 2 Simon Hohl 2018-12-18 08:45:52 UTC
It seems like it is caused when applying my matching rule:

Threshold: 1
Type: authority

Search index: identifier-standard
Score: 1

Tag: 024
Subfields: 9
Offset: 0
Length: 0

Oddly, running it again now gives me a whopping 139 occurrences of the warning (spamming the log and the reason why I'd like to get rid of the warning):

perl /usr/share/koha/bin/stage_file.pl --match 3 --file gazetteer_authority.mrc --authorities
... staging MARC records -- please wait
... finished staging MARC records
... looking for matches with records already in database
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in regexp compilation at /usr/share/perl5/MARC/Record.pm line 206.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 207.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
Use of uninitialized value $tag in hash element at /usr/share/perl5/MARC/Record.pm line 202.
... finished looking for matches

MARC record staging report
------------------------------------
Input file:                 gazetteer_authority.mrc
Record type:                auth
Number of input records:    31
Number of valid records:    31
Number of invalid records:  0
Number of records matched:  31
Comment 3 Jonathan Druart 2018-12-18 14:14:45 UTC
I do not have this index set for authorities:

oAuth error: Unsupported Use attribute (114) identifier-standard Bib-1

Same if I try Identified-Standard, as well as Local-Number...
Comment 4 Katrin Fischer 2018-12-18 14:26:15 UTC
Identifier-standard is not defined as an index for authority data, only for bibliographic. So I think it can't work here. But it should work for authority indexes like Local-Number. Not sure what's wrong here.
Comment 5 Simon Hohl 2018-12-18 14:55:17 UTC
Katrin is right, it is "Identifier-standard", uppercase i, lowercase s - sorry. 

I did not change my Zebra indexing configuration, the matching definitely works with my setup, weird. You are both unable to match with that rule?
Comment 6 Jonathan Druart 2018-12-18 15:06:14 UTC
oAuth error: Unsupported Use attribute (114) Identifier-standard Bib-1
Comment 7 Simon Hohl 2018-12-18 16:02:46 UTC
My version info:

Koha version: 	18.05.05.000
OS version ('uname -a'): 	Linux (..) 4.9.0-8-amd64 #1 SMP Debian 4.9.110-3+deb9u6 (2018-10-08) x86_64
Perl interpreter: 	/usr/bin/perl
Perl version: 	5.024001
Perl @INC: 	/usr/share/koha/lib
/etc/perl
/usr/local/lib/x86_64-linux-gnu/perl/5.24.1
/usr/local/share/perl/5.24.1
/usr/lib/x86_64-linux-gnu/perl5/5.24
/usr/share/perl5
/usr/lib/x86_64-linux-gnu/perl/5.24
/usr/share/perl/5.24
/usr/local/lib/site_perl
/usr/lib/x86_64-linux-gnu/perl-base
MySQL version: 	mysql Ver 15.1 Distrib 10.1.26-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2
Apache version: 	Server version: Apache/2.4.25 (Debian)
Memcached: 	Servers: 127.0.0.1:11211 | Namespace: (..) | Status: running. | Config read from: koha-conf.xml
Zebra version: 	Zebra 2.0.59 (C) 1994-2014, Index Data Zebra is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. SHA1 ID: c00bfddbf0f3608340d61298acc61dafb167f9b2 Using ICU
Date and time: 	18/12/2018 17:01
Time zone: 	Used: Europe/Berlin | Config: Undefined | Environment (TZ): Undefined
Comment 8 Simon Hohl 2019-05-13 08:00:57 UTC
Damn, I updated Koha and now I get the oAuth error after Re-Indexing.
Comment 9 Katrin Fischer 2019-05-13 10:53:56 UTC
(In reply to Simon Hohl from comment #8)
> Damn, I updated Koha and now I get the oAuth error after Re-Indexing.

Hi Simon, the mailing list would be better to discuss this as a lot more people are reading there. Please make sure to include your exact Koha version, how it was installed and of course a step by step to the error with the error message.
Comment 10 Simon Hohl 2019-06-03 13:27:26 UTC
I am back to the original situation (working, but with warnings):

1. Added to authority-koha-indexdefs.xml (roughly copied over from the biblio equivalent):
  <!--record.abs line 60: melm 024        Identifier-standard-->
  <kohaidx:index_subfields tag="024" subfields="abcdefghjklmnopqrstvxyz0123456789">
    <kohaidx:target_index>Identifier-standard:w</kohaidx:target_index>
    <kohaidx:target_index>Identifier-standard:p</kohaidx:target_index>
    <kohaidx:target_index>Identifier-standard:s</kohaidx:target_index>
  </kohaidx:index_subfields>
2. Added to etc/zebradb/authorities/etc/bib1.att:
  att 1007 Identifier-standard
3. Ran xsltproc on xml to create xsl file:
  xsltproc /etc/koha/zebradb/xsl/koha-indexdefs-to-zebra.xsl /etc/koha/zebradb/marc_defs/marc21/authorities/authority-koha-indexdefs.xml > /etc/koha/zebradb/marc_defs/marc21/authorities/authority-zebra-indexdefs.xsl
4. Rebuilt Zebra:
  koha-rebuild-zebra -a -f -v (...)
Comment 11 Katrin Fischer 2023-09-23 08:19:31 UTC
Hi Simon, does this problem still persist?
Comment 12 Simon Hohl 2023-09-25 06:39:22 UTC
We switched to ElasticSearch for some time, that "resolved" the warnings. ;)