Bug 18927 - koha-rebuild-zebra is failing with "error retrieving biblio"
Summary: koha-rebuild-zebra is failing with "error retrieving biblio"
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Command-line Utilities (show other bugs)
Version: master
Hardware: All All
: P5 - low critical (vote)
Assignee: Jonathan Druart
QA Contact: Marcel de Rooy
URL:
Keywords:
: 19101 (view as bug list)
Depends on:
Blocks: 17600 19687
  Show dependency treegraph
 
Reported: 2017-07-11 13:50 UTC by Jef Bucas
Modified: 2019-10-14 19:56 UTC (History)
12 users (show)

See Also:
Change sponsored?: ---
Patch complexity: Small patch
Text to go in the release notes:
Version(s) released in:


Attachments
Bug 18927: Use fully qualified subroutine names in C4::Items (8.27 KB, patch)
2017-09-04 15:10 UTC, Jonathan Druart
Details | Diff | Splinter Review
Bug 18927: Use fully qualified subroutine names in C4::Items (8.31 KB, patch)
2017-09-05 13:13 UTC, Lee Jamison
Details | Diff | Splinter Review
Bug 18927: Use fully qualified subroutine names in C4::Items (8.41 KB, patch)
2017-09-06 11:21 UTC, Marcel de Rooy
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Jef Bucas 2017-07-11 13:50:25 UTC
Using Debian packages:
koha:~# dpkg -l | grep koha
ii  koha-common                           17.05.01-1                         all          integrated (physical) library management system
ii  libcgi-session-driver-memcached-perl  0.04-1koha2                        all          Perl module to allow CGI sessions to be stored in memcache
ii  libconvert-basen-perl                 0.01-1koha1                        all          perl module for encoding and decoding of base{2,4,8,16,32,64} strings
ii  libdbix-runsql-perl                   0.14-1~koha1                       all          module to run an SQL file via a DBI handle
ii  libjson-validator-perl                0.67+dfsg-1~koha1                  all          module to validate data against a JSON schema
ii  libmarc-record-perl                   2.0.6-1koha1                       all          Perl extension for handling MARC records
ii  libmojolicious-perl                   6.15+dfsg-1~koha1                  all          simple, yet powerful, Web Application Framework
ii  libpdf-fromhtml-perl                  0.31-1~koha1                       all          module to convert HTML documents to PDF
ii  libpdf-reuse-perl                     0.36-1koha1                        all          Reuse and mass produce PDF documents
ii  libpdf-writer-perl                    0.06-1~koha1                       all          PDF writer abstraction layer
ii  libswagger2-perl                      0.77-1~koha1                       all          Perl module for generating, parsing and transforming swagger API documentation
ii  libtemplate-plugin-htmltotext-perl    0.03-1koha1                        all          Plugin interface to HTML::FormatText
ii  libtemplate-plugin-json-escape-perl   0.2-1koha1                         all          module for embedding JSON strings in Template Toolkit
ii  libtemplate-plugin-stash-perl         1.006-1~koha1                      all          Template::Toolkit plugin that exposes the template's stash



# koha-rebuild-zebra -f -v koha
returns a lot of errors:
[...]
error retrieving biblio 94540 at /usr/share/koha/bin/migration_tools/rebuild_zebra.pl line 683, <DATA> line 751.
error retrieving biblio 94541 at /usr/share/koha/bin/migration_tools/rebuild_zebra.pl line 683, <DATA> line 751.
error retrieving biblio 94542 at /usr/share/koha/bin/migration_tools/rebuild_zebra.pl line 683, <DATA> line 751.
error retrieving biblio 94543 at /usr/share/koha/bin/migration_tools/rebuild_zebra.pl line 683, <DATA> line 751.
error retrieving biblio 94544 at /usr/share/koha/bin/migration_tools/rebuild_zebra.pl line 683, <DATA> line 751.
error retrieving biblio 94545 at /usr/share/koha/bin/migration_tools/rebuild_zebra.pl line 683, <DATA> line 751.
error retrieving biblio 94546 at /usr/share/koha/bin/migration_tools/rebuild_zebra.pl line 683, <DATA> line 751.
error retrieving biblio 94547 at /usr/share/koha/bin/migration_tools/rebuild_zebra.pl line 683, <DATA> line 751.
error retrieving biblio 94548 at /usr/share/koha/bin/migration_tools/rebuild_zebra.pl line 683, <DATA> line 751.
error retrieving biblio 94549 at /usr/share/koha/bin/migration_tools/rebuild_zebra.pl line 683, <DATA> line 751.
[...]


Although the 'use' is clearly defined:
use C4::Biblio;

specifying the full perl library/function name in those 2 files fixes it:
for p in /usr/share/koha/bin/migration_tools/rebuild_zebra.pl /usr/share/koha/lib/C4/Items.pm; do
for f in GetMarcFromKohaField GetMarcBiblio GetBiblionumberFromItemnumber TransformKohaToMarc GetFrameworkCode; do
sed -i -e "s/$f/C4::Biblio::$f/g" $p
done
done
Comment 1 Claudio 2017-08-17 17:02:21 UTC
Jef,

I have the same problem in a production installation. I need your opinion about to follow your suggestion to fix it. I've open bz #19101 because a similar problem.
Comment 2 Claudio 2017-08-18 14:38:27 UTC
Jef,

The problem was fixed in my installation aplying your suggestion. I think this bug is not an enhancement but a major or critical one.
Comment 3 Katrin Fischer 2017-08-18 15:25:52 UTC
*** Bug 19101 has been marked as a duplicate of this bug. ***
Comment 4 Jonathan Druart 2017-09-04 15:10:47 UTC
Created attachment 66823 [details] [review]
Bug 18927: Use fully qualified subroutine names in C4::Items

rebuild_zebra.pl fails in some conditions (perl version?)
I do not recreate but it has been reported that reindex fails with:
  error retrieving biblio 94540 at /usr/share/koha/bin/migration_tools/rebuild_zebra.pl line 683, <DATA> line 751.

To fix it we can use fully qualified subroutine names for:
  GetMarcFromKohaField
  GetMarcBiblio
  GetBiblionumberFromItemnumber
  TransformKohaToMarc
  GetFrameworkCode

Test plan:
Confirm the rebuild_zebra script still works correctly after this patch
Comment 5 Lee Jamison 2017-09-05 13:13:34 UTC
Created attachment 66850 [details] [review]
Bug 18927: Use fully qualified subroutine names in C4::Items

rebuild_zebra.pl fails in some conditions (perl version?)
I do not recreate but it has been reported that reindex fails with:
  error retrieving biblio 94540 at /usr/share/koha/bin/migration_tools/rebuild_zebra.pl line 683, <DATA> line 751.

To fix it we can use fully qualified subroutine names for:
  GetMarcFromKohaField
  GetMarcBiblio
  GetBiblionumberFromItemnumber
  TransformKohaToMarc
  GetFrameworkCode

Test plan:
Confirm the rebuild_zebra script still works correctly after this patch

Signed-off-by: Lee Jamison <ldjamison@marywood.edu>
Script works correctly after patch is applied.
Comment 6 Marcel de Rooy 2017-09-06 11:21:34 UTC
Created attachment 66869 [details] [review]
Bug 18927: Use fully qualified subroutine names in C4::Items

rebuild_zebra.pl fails in some conditions (perl version?)
I do not recreate but it has been reported that reindex fails with:
  error retrieving biblio 94540 at /usr/share/koha/bin/migration_tools/rebuild_zebra.pl line 683, <DATA> line 751.

To fix it we can use fully qualified subroutine names for:
  GetMarcFromKohaField
  GetMarcBiblio
  GetBiblionumberFromItemnumber
  TransformKohaToMarc
  GetFrameworkCode

Test plan:
Confirm the rebuild_zebra script still works correctly after this patch

Signed-off-by: Lee Jamison <ldjamison@marywood.edu>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Comment 7 Marcel de Rooy 2017-09-06 11:22:53 UTC
Workaround. Looks good to me. But we still do not know what disturbs perl somewhere in those C4 modules..
Comment 8 Jonathan Druart 2017-09-06 16:32:21 UTC
Pushed to master for 17.11, thanks to everybody involved!
Comment 9 Fridolin Somers 2017-09-12 14:45:21 UTC
Depends on Bug 19040 but I can easily adapt for 17.05.x
Comment 10 Fridolin Somers 2017-09-12 14:54:19 UTC
Pushed to 17.05.x, will be in 17.05.04.
Comment 11 Katrin Fischer 2017-09-16 06:53:10 UTC
This patch has been pushed to 16.11.x and will be in 16.11.12.
Comment 12 jfowler 2017-09-21 21:08:25 UTC
How would I go about applying this particular patch? I just ran into this on a production instance.
Comment 13 Jonathan Druart 2017-09-21 21:21:39 UTC
(In reply to jfowler from comment #12)
> How would I go about applying this particular patch? I just ran into this on
> a production instance.

If you are using 16.05.x, 16.11.x or 17.05.x, you just need to upgrade using apt.
Comment 15 Ian Bays 2017-11-22 20:17:43 UTC
After recent upgrade to 17.05.04.000 bulkmarcimport started to fail on all items attempted to be added.

ERROR: Adding items to bib 1 failed: Undefined subroutine &C4::Items::TransformMarcToKoha called at /usr/share/koha/lib/C4/Items.pm line 382, <GEN46> line 1

So I manually added the prefix and the next error came up:

ERROR: Adding items to bib 74925 failed: Undefined subroutine &C4::Items::GetMarcStructure called at /usr/share/koha/lib/C4/Items.pm line 2307, <GEN46> line 14.

Manually adding the prefix to that line seems to have worked, but it does not feel like the right solution.
Comment 16 Mason James 2017-11-23 21:54:38 UTC
(In reply to Ian Bays from comment #15)
> After recent upgrade to 17.05.04.000 bulkmarcimport started to fail on all
> items attempted to be added.

i created a separate bug for this... bug 19687
Comment 17 Mason James 2017-11-23 21:57:16 UTC
(In reply to Katrin Fischer from comment #11)
> This patch has been pushed to 16.11.x and will be in 16.11.12.

Pushed to 16.05.x, for 16.05.17 release
Comment 18 David Cook 2018-10-30 04:31:05 UTC
This is an interesting one... 

I have the exact same Koha code running on Debian Jessie and OpenSUSE. The code on OpenSUSE works fine, but the code on Debian is creating the error reported in this bug.

Debian is Perl 5.20.2 using Koha apt repository and Debian apt repo for dependencies.

Exporter is 5.71. 

OpenSUSE is Perl 5.18.2 using CPAN for dependencies.

Exporter is 5.68.

--

Doesn't look like there's anything important in the Exporter changelog: https://metacpan.org/changes/distribution/Exporter. 

--

Hmm, the C4::Items::Item2Marc function looks exactly the same in 17.05.xx as it does in older versions of Koha that worked fine on Debian. 

When I try the older Koha library code on Debian with a little script I've written, it works fine (notwithstanding expected database schema errors). It's an issue with the newer Koha library code...

The exports look basically the same in C4::Biblio between versions. 

--

So it looks like something we must have done... yet it also looks like it depends on the system Perl too. Going to try a different Perl on OpenSUSE and see if that makes a difference...
Comment 19 David Cook 2018-10-30 06:11:28 UTC
(In reply to Marcel de Rooy from comment #7)
> Workaround. Looks good to me. But we still do not know what disturbs perl
> somewhere in those C4 modules..

This is what is really bothering me...

In Item2Marc, I'm doing the following:

my @keys = keys %C4::Items::;
use Data::Dumper;
warn Dumper(\@keys);

This outputs the following:

$VAR1 = [
          'ModClassSource',
          'getFacets',
          'ModClassSortRule',
          'getnbpages',
          'GetLogs',
          'GetMarcStructure',
          'GetItemnumberFromBarcode',
          'AddItem',
          'b',
          'ItemSafeToDelete',
          'GetLatestAcquisitions',
          'GetSourcesForSortRule',
          'GetVariationsOfISSN',
          'GetItemnumbersForBiblio',
          'MoveItemFromBiblio',
          'ShelfToCart',
          'GetItemsByBiblioitemnumber',
          'GetItemsLocationInfo',
          'Dumper',
          'ISA',
          'get_itemnumbers_of',
          'getitemtypeinfo',
          'GetNormalizedOCLCNumber',
          'ModZebra',
          'CartToShelf',
          'GetClassSortRules',
          'GetVariationsOfISBN',
          'ToggleNewStatus',
          'ModItemTransfer',
          'ModItem',
          'GetFrameworkCode',
          'output_pref',
          'import',
          'GetItem',
          'GetItemInfosOf',
          '_parse_unlinked_item_subfields_from_xml',
          '__ANON__',
          'GetClassSources',
          'CheckItemPreSave',
          '_set_derived_columns_for_mod',
          'getitemtypeimagedir',
          'xml_escape',
          'BEGIN',
          '_repack_item_errors',
          'AddClassSource',
          '_find_value',
          'GetMarcBiblio',
          'cronlogaction',
          '_do_column_fixes_for_mod',
          'any',
          '_set_derived_columns_for_add',
          'dt_from_string',
          '_set_defaults_for_add',
          'Item2Marc',
          '_build_default_values_for_mod_marc',
          'GetAnalyticsCount',
          'GetNormalizedEAN',
          '_mod_item_dates',
          'displaylog',
          '_get_single_item_column',
          'getitemtypeimagesrc',
          '_get_unlinked_subfields_xml',
          'GetHiddenItemnumbers',
          'GetNormalizedUPC',
          'get_hostitemnumbers_of',
          'GetNormalizedISBN',
          'SearchItemsByField',
          'EXPORT',
          'GetPrinter',
          'DEBUG',
          'a',
          'GetVariationsOfISBNs',
          'ModDateLastSeen',
          'Load',
          'AddItemFromMarc',
          'GetLastAcquisitions',
          'GetItemsInfo',
          'carp',
          'TransformKohaToMarc',
          'GetHostItemsInfo',
          '_get_unlinked_item_subfields',
          'GetMarcItem',
          'ModItemFromMarc',
          'GetClassSort',
          'DelItem',
          'GetItemTypesCategorized',
          'AUTOLOAD',
          'format_sqldatetime',
          'GetBarcodeFromItemnumber',
          'GetLogStatus',
          'getallthemes',
          '_marc_from_item_hash',
          'AddClassSortRule',
          'getitemtypeimagelocation',
          '_calc_items_cn_sort',
          'GetPrinters',
          '_koha_delete_item',
          'SearchItems',
          'DelClassSource',
          'GetClassSortRule',
          'GetVariationsOfISSNs',
          '_SearchItems_build_where_fragment',
          'NormalizeISBN',
          'TransformMarcToKoha',
          'GetClassSource',
          'NormalizeISSN',
          'GetMarcFromKohaField',
          'DelItemCheck',
          'GetAuthorisedValues',
          'confess',
          'GetLostItems',
          'PrepareItemrecordDisplay',
          'DelClassSortRule',
          '_koha_modify_item',
          'AddItemBatchFromMarc',
          'logaction',
          'croak',
          'IsMarcStructureInternal',
          '_koha_new_item',
          'GetItemsForInventory'
        ];

TransformKohaToMarc appears in that list. 

I can use "carp" or "C4::Items::carp", but I can't use "TransformKohaToMarc" or "C4::Items::TransformKohaToMarc". 

But the function seems to have been imported...