Bug 14858 - misc/batchRebuildItemsTables.pl has hard coded record length limit of 99999 bytes.
Summary: misc/batchRebuildItemsTables.pl has hard coded record length limit of 99999 b...
Status: CLOSED FIXED
Alias: None
Product: Koha
Classification: Unclassified
Component: Command-line Utilities (show other bugs)
Version: master
Hardware: All All
: P5 - low trivial (vote)
Assignee: Barton Chittenden
QA Contact: Testopia
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-09-18 14:53 UTC by Barton Chittenden
Modified: 2016-06-21 21:38 UTC (History)
5 users (show)

See Also:
Change sponsored?: ---
Patch complexity: ---
Bot Control: ---
When did the bot last check this:
Who signed the patch off:
Text to go in the release notes:


Attachments
Bug 14858: removed record length limit test (1.24 KB, patch)
2015-09-21 17:05 UTC, Barton Chittenden
Details | Diff | Splinter Review
Bug 14858: removed record length limit test (1.29 KB, patch)
2015-09-29 00:24 UTC, Nick Clemens
Details | Diff | Splinter Review
Bug 14858: removed record length limit test (1.36 KB, patch)
2015-10-01 14:10 UTC, Jonathan Druart
Details | Diff | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Barton Chittenden 2015-09-18 14:53:36 UTC
At line 73 of misc/misc/batchRebuildItemsTables.pl, we're skipping records whose $reclen is greater than 99999 bytes.


     if ($reclen > 99999) { push @errors,  "bad record length for biblionumber $biblionumber (length : $reclen) "; next; }

The 99999 byte limit is only valid for GRS-1, which has been removed from koha, therefore this limit should be removed.
Comment 1 Barton Chittenden 2015-09-21 17:05:42 UTC Comment hidden (obsolete)
Comment 2 Barton Chittenden 2015-09-21 17:37:35 UTC
Test plan:

Import / Create a MARC record that is more than 99999 bytes (this can be done by creating many items; the 952 fields will bulk up the record).

Manipulate the MARC on the large record so that some item fields (bulkmarcimport is a good candidate for creating and manipulating such a record).

Run 

misc/batchRebuildItemsTables.pl, and note that item fields are not updated for the jumbo record.

Apply the patch.

Re-run misc/batchRebuildItemsTables.pl, and see that the item fields now match the jumbo record.
Comment 3 Nick Clemens 2015-09-29 00:24:42 UTC Comment hidden (obsolete)
Comment 4 Nick Clemens 2015-09-29 00:28:37 UTC
Test plan was unclear, but here's what I did:

1 - Created a super big record (pasted a project gutenberg work into 520$b)
2 - Added items with $x - Non public and $z Public 
3 - Mapped field x to items.itemnotes_nonpublic
4 - Checked mysql table, items.itemnotes_nonpublic was NULL
5 - Ran batchRebuildItemsTables.pl - received an error
6 - Checked mysql table, items.itemnotes_nonpublic was NULL
7 - Applied patch
8 - Ran batchRebuildItemsTables.pl - no error
9 - Checked mysql table, items.itemnotes_nonpublic was 'Non public'
10 - Signed off
Comment 5 Jonathan Druart 2015-10-01 14:10:22 UTC
Created attachment 43013 [details] [review]
Bug 14858: removed record length limit test

Signed-off-by: Nick Clemens <nick@quecheelibrary.org>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Comment 6 Jonathan Druart 2015-10-01 14:11:14 UTC
Thanks for the test plan Nick!
Comment 7 Tomás Cohen Arazi 2015-10-06 14:07:14 UTC
Patch pushed to master.

Thanks Barton!
Comment 8 Chris Cormack 2015-10-23 13:58:46 UTC
Pushed to 3.20.x will be in 3.20.5