Bug 11399

Summary: batch rebuild items tables
Product: Koha Reporter: Stéphane Delaune <stephane.delaune>
Component: Command-line UtilitiesAssignee: Stéphane Delaune <stephane.delaune>
Status: CLOSED FIXED QA Contact: Testopia <testopia>
Severity: enhancement    
Priority: P5 - low CC: bgkriegel, katrin.fischer, kyle, tomascohen
Version: Main   
Hardware: All   
OS: All   
See Also: http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14858
Change sponsored?: --- Patch complexity: ---
Documentation contact: Documentation submission:
Text to go in the release notes:
Version(s) released in:
Bug Depends on:    
Bug Blocks: 13023    
Attachments: proposed patch
proposed patch
[SIGNED-OFF] Bug 11399: batch Rebuild Items Tables
followup : QA fixes
[PASSED QA] Bug 11399: batch Rebuild Items Tables
[PASSED QA] Bug 11399: QA fixes

Description Stéphane Delaune 2013-12-16 13:10:39 UTC
This script rebuilds the non-MARC DB items table from the MARC values (eg. in 995 for unimarc, 952 in marc21). You can/must use it when you change your mapping.

This script is similar to batchRebuildBiblioTables(which works on biblio an biblioitems table), but works on items only.
Comment 1 Stéphane Delaune 2013-12-16 13:20:08 UTC Comment hidden (obsolete)
Comment 2 Katrin Fischer 2013-12-17 09:15:24 UTC
Hi Stephane, I am a bit confused about the use of this, because I think we don't keep the item information in the MARC any more?
Comment 3 Stéphane Delaune 2013-12-17 09:48:31 UTC
(In reply to Katrin Fischer from comment #2)
> Hi Stephane, I am a bit confused about the use of this, because I think we
> don't keep the item information in the MARC any more?

Hi Katrin, in fact, informations about items aren't stored directly in MARC. However, when editing a record, MARC items informations can be regenerated temporarily (that's what serves '1' when we call the function 'GetMarcBiblio($biblionumber,1)'). When a mapping is changed in "MARC bibliographic framework" or "Koha to MARC mapping" (admin/koha2marclinks.pl on items), it does not change the existing items. For example in UNIMARC, without this patch, if I want to link the 995$j with items.stocknumber, this information will remain in items.more_subfields_xml for old items as long as they have not been modified manually. This script forces rewriting of items taking new mapping into account (in our example, all 995$j will then be moved from items.more_subfields_xml to items.stocknumber).
Comment 4 Katrin Fischer 2013-12-17 10:52:19 UTC
Hi Stephane, that makes sense - maybe we should change the bug description? I think this will only be needed if you change the mapping from a before unmapped field or when you map an unmapped one (more_subfields_xml)
Comment 5 Stéphane Delaune 2013-12-17 11:50:56 UTC Comment hidden (obsolete)
Comment 6 Bernardo Gonzalez Kriegel 2014-06-23 01:02:48 UTC Comment hidden (obsolete)
Comment 7 Katrin Fischer 2014-07-06 20:43:25 UTC
Comment on attachment 29156 [details] [review]
[SIGNED-OFF] Bug 11399: batch Rebuild Items Tables

Review of attachment 29156 [details] [review]:
-----------------------------------------------------------------

2 small things from code review:

::: misc/batchRebuildItemsTables.pl
@@ +1,4 @@
> +#!/usr/bin/perl
> +use 5.10.0;
> +use strict;
> +use warnings;

Please switch to: use Modern::Perl; :)

@@ +94,5 @@
> +    }
> +}
> +
> +$dbh->do("UPDATE systempreferences SET value=$CataloguingLog WHERE variable='CataloguingLog'");
> +$dbh->do("UPDATE systempreferences SET value=$dontmerge where variable='dontmerge'");

Please switch to using placeholders instead of variables.
Comment 8 Stéphane Delaune 2014-09-11 08:42:53 UTC Comment hidden (obsolete)
Comment 9 Kyle M Hall 2014-10-17 14:52:03 UTC
Created attachment 32511 [details] [review]
[PASSED QA] Bug 11399: batch Rebuild Items Tables

You can/must use it when you change items mapping.
this script rebuilds the non-MARC DB items table from the MARC values

usage : batchRebuildItemsTables.pl [ -h ][ -c ][ -t ][ -where ]

Options:
   -h --help       (or without arguments) shows this help message
   -c              Confirm: rebuild non marc DB (may be long)
   -t              test only, change nothing in DB
   --where         add where condition on default query
                   (eg. -where 'biblio.biblionumber<100')

(copied test plan)
to test it :

create a test item (eg on biblionumber 3) with a non mapped subfield with a test value.
check real items DB row ( SELECT * from items where biblionumber=3 ) => non mapped value is in more_subfields_xml
map the non mapped subfield with unused items column
re-check in Db that nothing changes
run ./batchRebuildItemsTables.pl -c --where 'biblio.biblionumber=3'
check real items DB row ( SELECT * from items where biblionumber=3 ) => new mapped value is not in more_subfields_xml anymore but in the mapped column

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Work as described, no koha-qa errors.
Add a new item to record, save with value in unmapped subfield ('x')
Change mapping, run script, value on newly mapped column

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 10 Kyle M Hall 2014-10-17 14:52:10 UTC
Created attachment 32512 [details] [review]
[PASSED QA] Bug 11399: QA fixes

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Comment 11 Kyle M Hall 2014-10-17 14:53:35 UTC
Note, I found it a bit confusing that a test run required both -t *and* -c. I would expect each to be independent, and use of both would be the same as -t alone. However, we have no formal guidelines for this yet, so I do not feel it is proper to fail qa on this point.
Comment 12 Tomás Cohen Arazi 2015-04-23 16:11:44 UTC
Patches pushed to master.

Thanks Stephane!